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EXECUTIVE  SUMMARY 


Over  the  past  decade  there  have  been  an  ever-increasing  number  of 
embedded  computer  systems  (EGS's)  integral  to  DoD  systems.  EGS's  are 
characterized  as  highly  accurate,  reliable  and  programmable  devices 
which  attribute  greatly  to  system  performance,  capability  and  flexi- 
bility. The  programmable  feature  of  an  SGS  is  derived  through  its  soft- 
ware computer  programs.  This  softvfare,  because  of  the  relative  ease  with 
which  it  can  be  changed,  provides  tremendous  potential  for  maintaining 
system  performance  and  capability  current  with  continuinK  'hanges  in 
requirements. 

EGS  software  can  be  changed  to  correct  errors  and  deficiencies, 
add  new  capabilities  and  enhancements,  and  compensate  for  changes  and/or 
degradation  in  system  equipment.  These  types  of  change  requirements 
continue  throughout  the  life  of  the  weapon  system. 

This  report  addresses  a systematic  approach  for  implementing  EGS 
software  changes  after  system  deployment. 

It  concludes  that  software  provides  tremendous  flexibility  in  ] 

responding  to  system  problems  and  requirements  over  the  life  of  the 
system  provided  an  efficient,  effective  and  responsive  EGS  management 
and  support  system  is  established  which  (l)  is  planned  and  fully  co- 
ordinated with  the  weapon  system  integrated  logistics  support  plan; 

(2)  includes  a staff  of  highly  qualified  and  experienced  personnel 
and  a maintenance  and  modification  laboratory  equipped  with  general 
purpose  computing  equipment,  dyneimic  simulation,  data  acquisition 


and  system  equipment  mock-ups;  and  (3)  employs  a systematic  method 


for  developing  changes  which  includes  strong  program  management,  con- 
figuration management,  test  management  and  system  engineering. 

The  report  recommends  (l)  that  greater  emphasis  be  placed  on  educa- 
ting management  on  the  requirements  and  benefits  of  ECS  software  support; 
and  (2)  that  stronger  and  more  substantive  management  policies  and  support 
be  provided  in  acquiring  resources,  in  particular  personnel,  and  in  imple- 
menting ECS  software  support  plans. 
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SECTION  I 


INTRODUCTION 

PROBLEM » 

Embedded  Computer  System  (ECS)^  software  is  receiving  high  level  Depart- 
ment of  Defense  (DoD)  interest  and  attention  because  of  the  many  problems 
experienced  and  high  dollar  expenditures  incurred  during  their  development 
and  acquisition.  (8:2-6)  To  help  alleviate  these  problems,  new  directives 
and  regulations  have  been  promulgated  which  establish  policy  and  offer 
guidance  in  this  area,  e.g. , Department  of  Defense  Directive  ^000,20, 
Management  of  Computer  Resources  in  Ma.ior  Defense  Systems,  dated  26  April 
1076  and  Air  Force  Regulation  800-14  Volume  I,  Management  of  Computer  Re- 
sources in  Systems,  and  Volume  II,  Acquisition  and  Support  Procedures  for 
Computer  Resources  in  Systems,  dated  26  September  1975.  (l)i  (?)  As  a 
result,  better  planned  and  executed  software  development  programs  are  in 
effect,  as  evidenced  by  the  Air  Force  F-I6,  B-1  and  AWACS  systems. 

However,  a grey  area  still  remains  in  regards  to  efficient  and  effec- 
tive management  and  support  of  ECS  software  after  deployment.  Greater 
Interest,  attention,  guidance  and  visibility  must  be  focused  on  this  phase 
of  the  life  cycle  if  a viable  solution  is  to  be  achieved  for  each  of  the 
ECS  software  systems  entering  the  DoD  inventory. 


1.  "An  ECS  is  Integral  to  an  electronic  or  electromechanical  system  (for 
example,  combat  weapon  system,  tactical  system,  aircraft,  ship,  missile, 
spacecraft,  command,  control  and  communication  systems)  from  a design, 
procurement  and  operation  viewpoint".  (8:3) 

2.  This  notation  is  used  throughout  the  report  for  sources  of  quotations 
and  references.  The  first  number  is  the  source  listed  in  the  Bibliography. 
The  second  is  the  page(s)  in  the  reference. 
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PURPOSE: 


The  purpose  of  this  report  is  to  provide  visibility  and  offer  f^uidance 
for  the  manaf^ement  and  support  of  deployed  ECS  softvjare.  Attention  is 
focused  on  plannin{",  changes  and  modifications^,  configuration  manage- 
ment, testing,  support  resources,  and  organizational  structure  as  they 
a poly  to  software. 

To  accomplish  this,  the  author  has  researched  the  limited  documenta- 
tion covering  the  area,  and  has  drawn  heavily  on  his  seven  years  of  ex- 
perience in  pioneering  ECS  software  management  and  supoort  while  working 
for  the  Air  Force  Logistics  Command  on  the  F-111  aircraft  system. 

BACKGROUND; 

Over  the  past  decade,  there  have  been  an  ever-increasing  number  of 
ECS's  inte'i'al  to  DoD  systems.  SGS’s  are  characterized  as  programmable 
devices  exhibiting:  high  speed,  accuracy  and  reliability  in  performing 
computations,  making  logical  decisions,  establishing  priorities,  select- 
ing alternatives  and  exercising  control.  These  features  ha.ve  contributed 
to  the  achievement  of  DoD  systems  with  greater  perfomance,  capability 
and  flexibility. 


1.  The  term  "modification"  as  used  in  this  report  means  software  changes 
which  are  visible  to  the  weapon  system,  i.c.,  effect  performance,  capability, 
opera-tion,  function,  mode,  etc.,  while  "change"  is  used  to  nea.n  software 
changes  vrhich  correct  deficiencies,  optimize  computer  memory  and  timing, 
change  logic,  coding, ^etc.  modification  is  also  used  synonymously  with 
software  block  change‘s.  There  is  a fire  line  betv.-een  change  and  modifi- 
cation and  at  times  they  are  used  interchangeably. 

2.  A software  block  chanr e is  a collection  of  software  changes  (i.e., 
changes  with  no  harrUrare  impacts)  which  are  concurrently  developed  and 
integrated  into  the  baae]ine  computer  progra.m  as  a unit  change. 
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The  programmable  feature  of  most  EGS's  is  derived  through  its  soft- 
ware computer  programs.  The  software  provides  a tremendous  potential  for 
maintaining  weapon  system^  performance  and  capability  current  with  the 
continuing  chamges  in  requirements.  This  concurrency  has  far  exceeded 
the  capability  of  hardware  changes  in  terms  of  responsiveness  and  cost, 
as  illustrated  by  Figure  I-l.  The  data  is  from  the  F-111  program  and 
shows  cost  and  time  for  implementing  comparable  capabilities,  (additional 
off-set  aim  points  and  updated  weapon  ballistics)  through  hardware  on 
F-111  A/E  aircraft  and  through  software  on  the  F-111  d/F  aircraft. 

The  differences  are  quite  dramatic  and  had  the  software  changes  not 
been  part  of  a normal  software  annual  update,  i.e.,  block  change,  the 
differences  would  have  been  even  more  dramatic,  as  these  software  changes 
could  be  implemented  in  a matter  of  weeks. 

The  fundamental  difference  between  software  and  hardware  that  per- 
mits software  modifications  to  be  implemented  much  faster  and  cheaper 
is  that  software  does  not  go  through  a production  phase  and  requires  no 
modification  kits.  With  the  exception  of  documentation,  the  cost  and 
time  for  a software  change  is  primarily  consumed  in  developing  and  test- 
ing the  prototype  which  when  complete  can  be  immediately  sent  to  the  field 
for  operational  use. 

Changes  and  modifications  that  can  be  implemented  through  software 
cover  a wide  range,  from  correcting  a deficiency,  to  adding  a new  weapon 


1.  The  term  "weapon  system"  is  used  in  this  report  for  clarity  in  lieu 
of  "DoD  system"  or  "system".  It  is  fully  recognized  that  ECS  applies 
equally  to  other  types  of  DoD  systems,  as  previously  defined. 
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system  capability.  There  are  a continuing  number  of  requirements  that 
dictate  softwaore  chajiges  during  the  operational  phase  of  the  weapon  system 
life  cycle.  These  requirements  can  be  categorized  as: 

• Addition  and/or  change  in  system  capability 

• Deletion,  addition  and/or  change  in  operational  modes 

• Changes  in  operational  functions 

• Changes  in  weapons  and/or  weapon  ballistics 

• Changes  and/or  replacement  of  system  equipment 

• Addition  of  new  equipment 

< Correction  of  deficiencies 

• Corrections  to  prograimming  problems 

• Corrections  to  operations  and/or  operational  procedures 

• Corrections  to  equipment  problems 

• Compensation  for  equipment  degradation 

• Optimization  of  computer  memory  utilization 

• Optimization  of  computer  time  utilization 

After  weapon  system  deployment,  it  is  the  responsibility  of  the  DoD 
supporting  agency  to  see  that  responsive  action  is  taken  to  investigate 
ECS  software  change  requirements  and  to  implement  those  that  are  valid. 
Each  Service  is  taking  steps  to  provide  the  required  support.  The  trend 
is  to  develop  organic  capabilities,  i.e.,  DoD  in-house  efforts,  to  perform 
this  maintenance^  function.  This  trend  is  particularly  evident  for  air- 
craft software  where  organic  support  has  been  implemented  for  the  F-111, 


1.  For  the  purpose  of  this  report,  maintenance  includes  all  changes, 
modifications,  restructuring  or  recoding  of  the  ECS  software  for  whatever 
reason,  and  is  used  synonymously  with  ECS  software  support.  It  also  in- 
cludes other  ancillary  and  attendant  tasks  performed  on  the  software. 
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A-7  and  F-14  aircraft.  The  Air  Force  has  also  planned  organic  software 
support  for  the  F-4,  F-I5,  F-I6,  B-1,  AWACS  and  for  most  new  electronic 
and  communication  systems  entering  the  inventory.  Results  of  software 
support  studies  on  these  systems  have  shown  that  organic  software  support 
is  (1)  the  most  efficient,  effective  and  responsive  to  user  requirements, 
(2)  provides  the  best  guarantee  of  uninterrupted  support  over  the  life 
cycle  of  the  weapon  system,  and  (3)  provides  DoD  with  the  best  means  of 
positive  control  over  system  performance  and  capabilities. 


SECTION  II 


I 

i 

i 

PIANNIKG  j 

Plannirit":  for  ECS  software  nanaGement  and  support  must  start  early  I 

in  weapon  system  acquisition  development,  where  it  is  closely  coordinated  | 

t 

and  integrated  with  other  weapon  system  logistic  and  support  pleins,  and 
continues  throughout  the  weapon  system  life  cycle.  This  Section  addresses 
the  planning  phase  essential  for  determining  and  implementing  efficient 
and  cost  effective  ECS  software  management  and  support  approaches.  It 
looks  at  ECS  softvrare  support  requirements  and  alternatives,  and  reviews 
formal  planning  procedures. 

The  first  step  in  planning  a support  strategy  is  to  prepare  an  explicit 
problem  statement  which  includes  requirements;  viable  alternatives  for 

satisfying  requirements  (including  criteria  for  evaluation);  constraints;  j 

I 

and  assumptions.  For  ECS  software,  support  requirements  and  alternatives 
are  generally  functions  of  a generic  class  of  software,  e.g.,  aircraft 
operational,  shipboard  control,  electronic  warfare,  ground  communication, 
command  and  control,  etc,,  and  in  some  cases  they  are  functions  of  a parti- 
cular ECS  software  system.  Each  generic  class  of  software  has  its  om 
peculiar  characteristics,  hardware/sof tware  interfaces,  system  configura- 
tion, deployment  environment,  user/supporter  relationships,  and  policies,  1 

practices  and  procedures  governing  management  and  support.  Each  of  these 
must  be  analyzed  and  investigated  closely  in  determining  support  require- 
ments and  in  determining  the  best  alternative  for  satisfying  these  re- 
quirements, Support  alternatives  are  also  dependent  on,  and  constrained 
by,  the  availability  and  location  of  certain  essential  support  resources. 

These  resources  are  weapon  system  equipment,  (i.e.,  the  ECS  computers. 


and  the  sensors,  displays,  controls,  etc.,  which  interface  with  the  ECS 
computers ) ; and  ECS  engineers . 


r 

j 

f 

f 

i 
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i 

f 

i 
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Weapon  system  equipment  is  essential  for  software  validation,  i.e., 
system  integration,  system  performance  tests,  compatibility  tests,  etc. 

It  is  normally  required  on  a dedicated  basis  and  is  configured  into  a 
system  test  bed.  The  location  and  availability  of  this  equipment  is  de- 
pendent on  the  type  of  weapon  system,  its  cost,  deployment  environment, 
the  number  of  systems  procurred,  and  its  maintenance  concept,  e.g. , on- 
site, depot,  contract,  etc.  Site  alternatives  for  ECS  software  support, 
i.e.,  maintenance,  will  typically  be  constrained  to  the  locations  where 
this  equipment  is  available. 

The  ECS  engineers  provide  the  technical  expertise  required  for  total 
system  (hardware/software)  performance,  integration,  compatibility  and 
configuration  and  are  typically  contractors  and/or  part  of  the  weapon 
system  support  agency  engineering  staff.  Their  availability  tends  to  be 
limited  to  particular  locations  and  this  can  have  an  impact  on  and/or 
constrain  the  choice  of  the  support  location  and/or  support  aigency. 

SUPPORT  REQUIREMENTS; 

ECS  software  support  requirements  can  be  divided  into  two  categories: 
(l)  ECS  software  change  and  modification  requirements,  and  (2)  routine 
software  maintenance  requirements. 

Software  change  and  modification  requirements  are  determined  in  terms 
of  change  frequency  and  change  response.  The  frequency  is  the  expected 
number  of  change  requirements  per  unit  of  time  (month,  year,  etc.)  and 
the  response  is  the  expected  time  for  transforming  requirements  into  re- 
leased changes.  Determining  frequency  and  response  is  not  simple  and  can 


o 


widely  vary  with  the  category  of  changes  (reference  Section  I).  However, 
if  the  changes  are  divided  into  two  classes:  (l)  software  only  changes 

(no  system  hardware  impacts)  and  (2)  software  changes  with  system  hardware 
impacts  or  caused  by  hardware  changes,  then  an  average  mean  frequency  and 
average  mean  response  for  each  of  these  classes  can  be  detemined  with 
reasonable  confidence. 

Routine  software  maintenance  requirements  are  determined  in  terras  of 
the  amount  of  activity  associated  with  software  optimization,  problem  in- 
vestigation, studies,  performance  analysis  and  response  to  day-to-day 
"what  if  situations" , 

The  aggregate  of  these  requirements  will  normally  dictate  a "sustained 
level  of  effort  support^  over  the  operational  phase  of  the  system  life 
cycle.  This  has  proven  to  be  the  case  for  support  of  the  FB-lllA,  F-111d/f 
and  A-?D/e  aircraft  Operational  Flight  Programs  (OFF’s)  . In  the  case  of 
the  F-111  OFP's,  the  software  change  activity  averages  about  2?  changes 
per  OFF  update,  i.e.,  block  change.  These  updates  run  continuously  for 
each  aircraft  configuration  and  take  15  - 18  months  for  development  and 
implementation.  This,  coupled  with  the  routine  mainteiiance  sunport  re- 
quirements, has  resulted  in  approximately  a 60  man  year  sustained  level 
of  effort. 


1.  "Sustained  level  of  effort  support"  means  a constant  staff  of  personnel 
(Government,  contractors,  or  mix)  employed  for  an  indefinite  period  of  time 
(generally  life  of  weanon  system)  to  perform  software  maintenance  (exclusive 
of  major  modifications  having  hardware  impacts). 

2.  OFP’s  are  the  software  resident  in  the  aircraft  avionics  computers  which 
integrate  the  on-board  sensors,  control  systems,  and  displays;  perform  com- 
putations and  exercise  control;  and  provide  operator  cues  for  automatic  and/( 
crew  aided  navigation,  bombing  and  fire  control. 
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ALTERNATIVES » 


Alternatives  for  ECS  software  maneigeinent  and  support  generally  consist 
of  some  combination  of  arrangements  between  the  weapon  system  support  agency, 
user  and  contractor  with  the  maintenance  site  being  either  the  depot,  a 
user  site,  or  contractors  facility.  For  example,  one  combination  might 
be  a situation  where  the  support  eigency  has  management  resi>onsibility,  the 
contractor  performs  the  maintenance  and  the  maintenance  facility  is  located 
at  the  user  site.  Alternatives  are  normally  constrained,  to  some  degree, 
by  Service  and  Command  regulations  and  policies  delineating  management  and 
support  responsibilities.  Key  criteria  used  for  evaluating  alternatives 
consist  of  cost,  availability  of  resources,  support  effectiveness,  ECS 
software  control  and  risk  associated  with  a continuous  support  posture.  An 
attempt  should  be  made  to  rank  the  criteria  in  order  of  importance.  How- 
ever, this  is  difficult  since  the  criteria  are  not  independent.  For  example, 
cost  is  highly  dependent  on  availability  of  resources. 

Very  little  latitude  exists  for  evaluating  alternatives  for  assign- 
ing management  responsibility  to  ECS  software.  Generally,  it  is  assigned 
to  the  organization  having  management  responsibility  for  the  weapon  system 
and/or  subsystem  containing  the  ECS.  For  example,  in  the  Air  Force,  air- 
craft OFP's  are  managed  by  the  aircraft  system  manager,  and  the  avionics 
subsystem  ECS  software  is  managed  by  the  respective  subsystem  mamager. 

There  is  much  more  latitude  for  evaluating  alternatives  for  the 
assignment  of  responsibility  for  ECS  software  support,  i.e.,  maintenance. 
However,  there  are  still  certain  constraints.  For  example,  maintenance 
for  command  and  control  software  and  software  relating  to  mission  require- 
ments is  traditionally  assigned  to  the  user.  The  evaluation  of  alternatives 
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for  EG3  software  raaintenance  is  one  of  the  more  difficult  and  emotional 
planning  tasks.  V/ith  the  increasing  number  of  EGS's  entering  the  inventory, 
software  maintenance  is  now  looked  unon  as  a very  attractive  and  prestigious 
workload  with  everyone  vranting  a piece  of  the  action.  The  evaluation  out- 
come tends  to  be  a function  of  who  performs  it.  Therefore,  an  evaluation 
performed  by  a teaja  made  up  of  all  involved  agencies  generally  produces 
a result  with  the  greatest  credence.  It  must  be  emphasized  that  each 
system  will  have  different  circuir.stances  and  must  be  evaluated  accordingly. 

Site  selection  for  a location  to  perform  softvjare  maintenance  is 
normally  constrained  to  sites  which  have  weapon  system  equipment  avail- 
able that  can  be  used  in  a system  test  bed  configuration.  This  constraint 
is  v(GP,pon  system  dependent.  For  exampl.e,  a one-of-a-kind  system  like  the 
FPS-85  satellite  tracking  radar  is  only  available  in  a system  configura- 
tion at  tho  user  site.  This  makes  that  site  the  only  practical  choice, 
as  the  site  for  software  maintenance.  In  this  situation,  software  main- 
tenance requirements  must  be  integrated  with  operational  requirements  and 
use.  The  alternative  is  to  develop  software  changes  at  a remote  location 
ai;d  bring  them  to  the  user  site  for  validation  and  system  test,  Kovfever, 
this  has  not  proven  cost  effective  because  of  the  iterative  nature  of 
software  development  and  test. 

In  contrast,  depots  for  fighter  aircraft  have  proven  to  be  tlie  best 
site  location  for  OFP  software  maintenance  support.  The  system  and  system 
equipment  are  more  readily  available  at  the  depot,  along  with  the  required 
technical  exwertise  and  other  support  resources.  Also,  if  the  aircraft 
are  deployed  at  more  than  one  cite  or  used  by  more  than  one  Command  or 
Seir/ice,  the  depot  becomes  the  logical  central  support  point.  Further, 
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fighter  aircraft  are  configured  in  such  a manner  that  field  softvrare 
maintenance  would  be  extremely  difficult.  The  equipment  is  densely 
packa^yed,  making  on-board  maintenance  impossible,  and  because  of  the 
field  maintenance  concept  for  the  electi'onics  equipment,  other  resources 
such  as  a laboratory  equipment  mock-up  are  not  available  in  the  field. 


PRCGHDUHaS! 

Formal  procedures  such  as  APR  800-14  have  been  instituted  to  assist 
personnel  in  the  difficult  task  of  planning  SGS  software  support.  AFR 
300-14,  Volume  II,  requires  that  a computer  resources  integrated  support 
nlan  (GRISl)  be  nrenared  for  each  SGS  software  system  entering  the  in- 
ventory, Early  in  system  full  scale  engineering  development,  a computer 
resotirces  worldnr  group  (GKV/G)  is  established  for  the  purpose  of  preparirg- 
the  GRISr  and  overseeing  its  implementation,  and  ha.s  responsibility  for 
determining  SGS  software  support  requirements  and  alternatives,  aiid  for 
coordinatin-';-  the  tra.ncitl  on  of  the  EGG  softvrare  from  the  developing  to 
the  support infg  and  using  agencies.  The  GRVJG  is  ma,dc  up  of  personnel  from 
the  developing,  supportinr,  tra.ininr',  and  using  agencies  and  also  has 
representa.tion  from  the  developin':'  contractors.  (lO-^^iS) 

The  GRIST  is  the  key  planning  document  for  SGS  software  management 
and  support,  it  is  prepared  early  in  full  scale  engineering  develop- 
ment, becomes  part  of  the  overall  wca’^on  system  Intcrrated  logistics 
support  plan  (ILST).  and  remains  updated  tb.rou/hout  the  system  life  cycle. 
The  CRISP  covers  all  mana  ement  and  support  aspects  essential  to  the 
maintenance  and  modification  of  EGG  softvrare  after  deployment,  including 
requirements  for  transfer  and  turnover  of  the  EGG  softvrare  from  the 
developing  to  the  supporting  and  using  a,ger.cles.  (li3“4,5) 


Based  on  expected  ECS  software  support  requirements  and  the  selected 
course  of  action  for  implementation,  the  CRISP  details  and  schedules  all 
activities,  functions,  and  resources  required  for  development,  implementa- 
tion and  operation  of  an  ECS  software  majiagement  and  support  capability. 
Included  in  the  CRISP  arej  (l)  organizationaJ.  responsibilities,  relation- 
ships, and  interfaces;  (2)  site  location  for  maintenance;  (3)  configuration 
management  plan;  (4)  modification  and  change  plein;  (5)  test  plan;  (6)  fund- 
ing plan;  (?)  hardware/sof tware  Integration  plan;  and  (8)  the  resource 
allocation  plan,  which  includes  requirements  and  sources  for  personnel 
and  training,  facilities,  laboratory  and  weapon  system  equipment,  computers, 
software,  and  documentation.  (ls3~^»5)»  (i^3) 

The  CRISP  also  integrates  the  ECS  software  transition  with  the  weapon 
system  program  management  responsibility  transfer  (PMRT)  by  closely  inter- 
facing with  the  PMRT  plan.  This  plan  establishes  a time  phased  schedule 
of  actions  and  events  necessary  for  accomplishing  an  orderly  and  timely 
transfer  of  weapon  system  program  management  responsibility  from  the  de- 
veloping agency  to  the  supporting  agency  on  a specific  date.  Figure  II-l 
shows  the  pre  PMRT  milestones  as  they  relate  to  the  CRISP.  (23) 

In  summary,  early,  thorough  and  fully  coordinated  planning  should 
result  in  the  selection  of  the  most  cost  effective  and  efficient  approach 
for  the  management  and  support  of  ECS  software  and  should  result  in  a 
smooth  transition  of  the  ECS  software  from  the  developing  to  the  support- 
ing and  using  agencies  with  a support  capability  in  place  and  operational 
at  the  time  of  transfer. 
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SECTION  III 


GHAI.'GE  PROCESS 

Soft’.', 'are  provides  the  vfeapon  system  vrlth  tremendous  flexibility 
because  of  the  rela,tive  ease  i-rith  imich  it  can  be  chc-nyed.  Hovfevcr,  the 
success  in  acliieving  weapon  system  flexibility  through  software  changes 
is  hif'hly  dependent  on  the  ability  to  mal;e  timely,  efficient,  and  effec- 
tive cha.nfes.  This  requires  an  orderly,  systematic  change  process,  com- 
bined vrith  considerable  reso'urce  commitments.  This  section  addresses 
the  technical  change  process.  It  examines  a typical  ECS  softvfare  chang'c 
a.nd  discusses  the  change  sequence,  block  change  concept  and  develonnent 
cycle.  Subsequent  sections  expand  on  conf igisration  nana  ement,  testing, 
resources  and.  the  support  organizo.tion. 

SCFT.fAfiE  C'lANGES ! 

As  noted  in  Section  I,  changes  to  EG3  softvjare  are  required  for  a 
n’Jir.bcr  of  reasons  and  v;ill  continue  over  the  life  of  the  vfeapon  system. 
As  the  weapon  system  matures,  the  reduced  number  of  changes  required  to 
correct  en;ors  and  deficiencies  is  off-set  by  the  increased  number  of 
cha.nges  required  to  implement  nevr  capabilities,  enha.ncements , etc. 
most  cha.rrcs  are  hinhly  complex,  as  depicted  in  Figure  III-l  and  III-?. 
Illustra.ted  is  the  "'■•'ind  Vector  Fix"  cha.rg-e  which  h?.s  been  impler.er.tcd 
in  F-111  CFF's,  a.nd  can  be  considered  a typical  ECS  softviare  chanre. 
Fi-ure  III-l  summarises  the  chan"c  requirement  in  terns  of  operational 
enhancements,  i.e.,  greater  naviration  a.nd  vfea.pon  delivery  accuracy  in 
the  de'radod  onerationa.1  mode  ar.d  easier  onera.tor  use,  Fi  urc  III-? 
illustra.tes  chan-  e mechanisa.tion  and  avionics  runctional  interfaces 
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IMPROVE  PROBABILITY  OF  MISSION  SUCCESS  IN  DEGRADED  MODE 


Figure  III-l  (17:14-4)  OFF  CHANGE  "UIND  VECTOR  FIX' 


DIGITAL  COMPUTER  COMPLEX 


aCHANIZATION/lNTEGRATIOM  "WIND  VECTOR  FIX" 


which  have  been  translated  from  the  chanpe  requirement. 
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To  effectively  make  changes  of  this  type  requires  oersonnel  who  have 
intimate  knowledge  of  the  weapon  system,  computer  interfaces,  avionics, 
and  ECS  software.  Further,  they  must  have  adequate  tools  to  analyze, 
process  and  test  software  changes.  These  resources  are  discussed  in 
detail  in  Section  VI  and  Appendix  B.  (l?) 


CHANGE  SEQUENCE; 

The  software  change  sequence  is  a process  of  analyzing,  designing, 
programming,  processing,  debugging,  integrating,  testing,  evaluating, 
and  implementing  changes  to  ECS  computer  programs.  This  process  is  illus- 
trated in  Figure  III-3  and  although  not  shown,  is  highly  iterative.  It 
begins  with  the  analysis  of  the  change  requirement  to  determine  validity, 
feasibility,  risk  and  scope.  The  analysis  includes  investigation  of: 


Operational,  performance,  functional  and  inter- 
face requirements 


• Computer  time,  core  and  word  length  requirements 

• Interfaces  and  weapon  system  impacts 

• Test  provisions  and  data 

• Documentation  requirements 

• Integration  and  acceptance  testing 

• Time,  cost  and  resource  requirements 

• Alternatives 

• Trade-off  considerations 

This  analysis  is  usually  performed  in  the  laboratory,  using  a dynamic 
simulator  and  system  mock-up  (reference  Section  VI  and  Appendix  B). 
Outputs  of  this  phase  are  a change  requirements  document  and  statement 


problems/requirskents 


r 


of  work.  Requirements  are  specified  in  terms  of  operational  performance 
and  capability. 

During  the  design  phase,  operational  requirements  are  translated 
into  interface,  performance,  and  functional  requirements.  These  require- 
ments are  then  translated  into  a detailed  design  which  reflects  program 
structure,  logic,  timing,  inputs,  outputs,  equations,  and  algorithms. 

The  output  of  this  phase  is  a design  specification  which  reflects  design 
requirements,  detailed  design,  and  acceptance  and  test  criteria. 

During  the  programming  phase,  the  detailed  design  is  translated  into 
a source  computer  program  (symbolic  code).  This  translation  is  performed, 
taking  into  account  computer  timing,  memory,  word  length,  accuracy,  and 
subroutine  interfaces. 

The  data  processing  phase  uses  a host  computer  to  assemble  or  compile 
the  source  program  into  an  object  program,  i.e.,  load  program  for  the  £GS 
computer.  In  addition,  documentation,  such  as  program  listings  and  memory 
maps  are  generated.  The  output  of  this  phase  is  the  ECS  computer  program. 

The  debug,  integration,  test  and  evaluation  phases  are  performed  both 
in  the  laboratory  and  in  an  operational  environment.  Computer  program  de- 
bugging is  performed  using  a host  computer,  dynamic  simulator  and  diagnostic 
software.  Integration,  test  and  evaluation  are  performed  using  a dynamic 
simulator,  system  mock-un  and  operational  weapon  system. 

The  final  phase  releases  the  ECS  computer  program  for  field  implementa- 
tion, This  is  accomplished  after  the  changes  have  been  documented  and 
accepted  by  the  user.  (3),  (13).  (l?) 
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BLOCK  GHAtlGE  CONCEPT; 


At  the  rate  which  ECS  software  changes  are  requested,  it  would  be 
highly  impractical  to  attempt  Implementation  on  an  individual  basis.  Most 
support  systems  could  not  be  responsive  or  afford  the  cost.  Major  problems 
would  be  created  in  allocating  and  scheduling  resources,  controlling  con- 
figuration and  maintaining  documentation.  Because  of  these  problems,  the 
Sacramento  Air  Logistics  Center  created  the  "block  change  concept"  for 
processing  changes  to  F-111  OFP's.  As  stated  in  Section  I,  a block  change 
is  a collection  of  software  changes,  i.e.,  changes  with  no  hardware  impacts, 
which  are  concurrently  developed  and  integrated  into  the  baseline  computer 
program  as  a unit  change.  Block  changes  are  normally  made  on  a periodic 
basis  with  the  cycle  time  negotiated  by  the  user  and  supporting  agency. 

It  depends  on  change  activity,  change  response  requirements  and  available 
support  resources.  For  example,  block  changes  are  made  to  F-111  OFP's 
every  12  to  18  months. 

Change  requests  are  analyzed  as  received,  with  feasible  changes  ac- 
cumulated as  block  change  candidates.  Emergency  changes  are  expedited 
by  processing  on  an  individual  basis.  Block  change  candidates  are  re- 
viewed on  a scheduled  basis  by  the  user  and  support  agency  for  the  purpose 
of  prioritizing  and  establishing  the  block  change  definition.  This  process 
is  discussed  in  detail  in  Section  IV  under  Configuration  Control. 

The  block  change  concept  enhances  the  efficiency,  effectiveness  and 
responsiveness  of  the  change  process  because  changes  can  be  collectively 
developed,  tested  and  documented.  The  major  pay-off  is  in  the  documenta- 
tion and  test  areas.  (l4),  (1?) 


CHANGE  DEVELOPMENT  CYCLE; 

The  ECS  software  change  development  cycle,  as  illustrated  in  Figure 
III-4  can  be  viewed  as  a microcosm  of  the  acquisition  development  cycle. 

It  provides  the  basis  for  an  orderly,  systematic  and  well  controlled  soft- 
ware update  program.  It  consists  of  a series  of  phases,  each  well  defined 
and  separated  by  significant  milestones.  The  cycle  was  established  for 
development  of  block  changes  to  OFP’s,  most  notably  F-111  OFP's,  but  is 
applicable  to  other  ECS  software. 

As  shown,  the  cycle  is  designed  to  allow  ample  time  for  testing  and 
documenting  changes.  These  are  major  time  consumers  and  start  at  the 
be' inning  and  continue  throughout  the  cycle  with  the  last  third  dedicated 
almost  exclusively  to  these  efforts.  (Only  a relative  time  scale  is  shown.) 
The  cycle  is  also  designed  to  provide  maximum  interface  and  communication 
between  the  user  and  developer. 

The  cycle  begins  with  a joint  user,  developer  meeting  to  review  and 
prioritize  change  requirements.  Results  of  the  analysis  performed  on 
previously  received  change  requests  (reference  Section  IV  and  Figure  IV-l) 
are  reviewed  to  determine  which  changes  should  be  included  in  the  block 
change  definition.  Trade-offs  are  made  between  user  priorities,  and  the 
scope  of  each  change  in  order  to  define  a block  change  which  can  be  com- 
pleted with  available  support  resources  and  within  the  development  cycle 
time.  The  meeting  is  concluded  with  a block  change  definition. 

The  review  is  followed  by  a continuation  of  the  requirement  analysis 
phase.  During  this  phase,  operational  requirements  are  translated  into 
design  requirements  and  a preliminary  detailed  design.  This  is  followed 
by  a preliminary  design  review  (PDR)  and  block  change  definition  meeting. 
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Computer  program  Change  Proposal 
Computer  Program  Configuration  Sub-Boaurd 


Here,  the  user  reviews  the  requlrenents  docunent  and  adequacy  of  design. 

The  design  is  revievred  to  assure  that  it  will  accomplish  intended  results, 
that  risk  has  heen  nininised  and  that  it  can  be  completed  on  time  and  v;ith 
allocated  resources.  The  outputs  of  this  review  are  the  block  charige  re- 
quirements docunent,  preliminary  design  and  agreement  to  pi'oceed  into 
deta-ilcd  design.  Also,  durinf'  this  phase  the  test  plc.n  and  documentation 
requirements  are  initiated.  (Reference  Sections  IV  and  V. ) 

Duririg  the  design  phase,  detailed  design  is  completed  and  the  computer 
program  change  proposal  (GPGP)  is  prepared  a,nd  submitted  to  the  computer 
program  confi'-uration  sub-boa.rd  (CPCSB)  for  approval.  (Ghanre  Control 
is  covered  in  Section  IV  under  Configuration  Control.)  The  GPGP  describes 
in  dcta.il  the  bloc!:  change,  total  weapon  system  impact,  and  cost  and 
schedule  for  imnlementation.  GPGSB  approval  constitutes  formal  "go  a.head" 
to  develop  th.e  block  change  and  implement,  subject  to  final  acceptance  of 
the  computer  nrogram  by  the  user. 

Lievelopment  commences  with  appi’oval  of  the  GPGP.  Here  the  cycle 
deviates  slightly  from,  the  a.cquisition  development  cycle  in  that  a critical 
design  review  (GDi;)  is  not  held  with  the  user  nrlor  to  proceeding  into 
design  implementation.  Durin,'-  development,  the  block  charirge  is  coded 
and  assembled  into  a.  new  engineering  ba-seline  nrogram.  liach  program 
cha.nge  and  the  fina.l  assembled  enginecrirg'  program  goes  throu'^h  extensive 
debugging,  system  integration  a.nd  test  in  the  laboratory.  Also  during 
this  phase,  the  tent  plan  and  formal  test  procedures  are  finalised  and 
documentation  is  concurrently  upda.ted  with  cha.nge  development  and  closely 
monitored  by  configur<ation  managonort. 
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At  completion  of  initial  development,  a GDR  and  user  demonstration 
is  held.  During  this  review,  the  design  is  examined  for  completeness 
and  each  change  is  demonstrated  to  the  user  to  verify  that  it  performs 
as  expected  in  a laboratory  environment.  Also  during  this  review,  design 
specifications,  test  plans  and  test  procedures  are  reviewed  for  adequacy, 
completeness  and  acceptability. 

Agreement  between  the  user  and  developer  at  the  GDR  that  development 
is  complete  constitutes  "go  ahead"  into  final  test  and  evaluation.  Dur- 
ing this  phase,  development  testing  continues  in  the  laboratory  and  the 
developer  integrates  and  conducts  developmental  tests  on  an  operational 
system.  For  example,  OFP's  are  tested  using  instrumented  flight  test 
aircraft.  Initial  testing:  of  the  ECS  software  integrated  with  an  opera- 
tional system  assures  that  no  major  problems  or  unsafe  conditions  exist. 
Once  this  is  accomplished,  the  ECS  software  is  certified  as  worthy  for 
operational  testing  and  is  released  to  the  user  for  test  purposes.  De- 
velopmental and  operational  testing  are  then  conducted  concurrently  and 
problems  and  anomalies  fed  back  to  the  laboratory  for  analysis  and  correc- 
tion. 

Part  way  through  the  test  program  a configuration  review  is  held. 

The  purpose  of  this  review  is  to  assess  test  results  and  freeze  configu- 
ration. At  this  point,  changes  experiencing  problems  which  do  not  have 
readily  apparent  solutions  that  can  be  implemented  and  tested  with  low 
risk  by  the  scheduled  completion  of  testing  are  removed  from  the  block 
change  and  deferred  to  the  next  update.  The  idea  behind  this  philosophy 
is  to  allow  sufficient  testing  of  the  final  configuration  and  allow  re- 
quired lead  time  for  final  documentation. 
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At  the  completion  of  scheduled  testing,  results  are  analyzed,  evalu-  i 

ated  auid  documented  in  the  final  test  report.  Results  are  reviewed  and 
the  user  makes  a determination  on  final  acceptance.  If  accepted,  the 
program,  along  with  required  field  documentation,  is  released  to  the  user 
for  operational  implementation  through  the  vreapon  system  release  process. 

During  this  phase,  all  documentation  and  testing  are  completed.  Also, 
the  fimctional  and  physical  configuration  audits  are  performed.  These 
audits  are  discussed  in  Section  IV  under  Configuration  Status  Accounting. 

(13),  (14),  (17) 

In  svunmary,  a methodology  for  efficient,  effective  and  responsive 
ECS  software  change  processing  was  discussed.  The  methodology  develops 
changes  collectively  as  a block  change  using  a development  cycle  tailored 
for  ECS  software  support. 
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SECTION  IV 


CONFIGURATION  MANAGEMENT 

Configuration  management  is  perhaps  the  least  understood,  and  most 
difficult  to  enforce,  of  all  disciplines  associated  with  the  management 
and  support  of  ECS  software,  yet  without  question,  it  is  one  of  the  most 
important.  Many  technical  personnel,  and  even  managers,  because  of  vague 
understanding,  perceive  configuration  management  as  a time  consuming  over- 
kill control  task  which  obstructs  the  software  change  process.  Yet, 
without  it,  programs  invariably  get  into  trouble.  For  example,  programmers 
tend  to  code  early  without  adequate  definition  or  design;  designs  tend  to 
get  implemented  without  adequate  documentation:  because  of  the  ease  v;ith 
which  software  can  be  changed,  unauthorized  and  unapproved  changes  tend 
to  get  implemented;  and  because  of  the  intangible  nature  of  software  (cannot 
see  or  touch  it),  these  changes  tend  to  go  undetected  until  they  create 
major  problems.  These  types  of  problems,  coupled  with  software  complexity, 
flexibility,  and  continuous  state  of  change,  during  the  operational  phase, 
make  it  imperative  that  all  personnel  working  with  ECS  software  understand 
and  employ  sound  configuration  manaigement  principles  and  practices. 

This  section  attempts  to  put  configuration  management  into  proper 
perspective  by;  (l)  reviewing  configuration  management  principles  and 
practices;  and  (2)  looking  at  how  they  apply  and  can  be  tailored  to  ECS 
software  support.  In  accordance  with  DoD  Directive  5000.29,  ECS  software 
will  be  managed  as  configuration  items  (CI's).  A GI  is  defined  as  "an 
aggregation  of  hardware/software  or  any  of  its  discrete  portions,  which 
satisfies  an  end  use  function  and  is  designated  by  the  Government  for 
configuration  management".  (5;2),  (?) 
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Configuration  management  provides  the  management  tools  and  procedures 
essential  for  systematic  ECS  software  maintenance  and  modification.  It 
is  defined  as  "a  discipline  applying  technical  and  administrative  direction 
and  surveillance  to:  (l)  identify  and  document  the  functional  and  physicaLI 

characteristics  of  a GI;  (2)  control  changes  to  those  characteristics; 
and  (3)  record  and  report  change  processing  and  implementation  status". 
(5:2)  This  definition  identifies  the  three  functions  of  configviration 
management : 

• configuration  identification  (for  software  the 
significance  is  on  identification  of  functional 
characteristics,  since  software  has  no  physical 
characteristics ) 

• configxxration  control 

• configuration  status  accounting 

GOtJFIGURATION  IDENTIFICATION: 

Configuration  identification  is  defined  as  "the  current  approved  or 
conditionally  approved  technical  documentation  for  a Cl,  as  set  forth  in 
specifications,  drawings  and  associated  lists,  and  documents  referenced 
therein",  and  configuration  is  defined  as  "the  functional  and/or  physical 
characteristics  of  haxdware/software,  as  set  forth  in  technical  documenta- 
tion and  achieved  in  a product".  (5*2)  For  the  purpose  of  this  report, 
CI*s  include  the  ECS  computer  programs  and  elements  of  the  ECS  software 
support  system  (reference  Section  VI  and  Appendix  B).  From  a weapon 
system  point  of  view.  Cl's  are  the  end  item  software,  i.e.,  the  ECS 
computer  programs.  However,  for  efficient  and  effective  ECS  software 
support,  it  is  essential  that  all  elements  of  the  ECS  software  support 


system,  both  hardware^  and  software,  also  be  mansiged  as  Cl's. 

The  primary  purpose  for  managing  the  ECS  software  and  support  system 
as  Cl's  is  to  assure  positive  control  over  their  configuration.  This  is 
required  in  order  to  efficiently  and  effectively  track  performance,  identify 
deficiencies,  and  effect  changes.  Hence,  the  documentation  must  be  tailored 
to  provide  a clear,  accurate,  thorough,  explicit  and  understandable  defini- 
tion of  the  software.  The  definition  must  define  the  computer  program 
requirements,  functions,  performance.  Inputs,  outnuts,  interfaces,  opera- 
tional characteristics,  and  structure.  Requirements  should  be  stated  in 
terms  of  operational  needs  or  deficiencies.  Functions,  performance,  inputs, 
outputs,  interfaces  and  operational  characteristics  should  reflect  the 
translation  of  requirements  into  engineering  design;  and  the  computer  pro- 
fjram  structure  should  reflect  the  detailed  software  design  and  include 
narratives,  timing,  memory  utilization,  logic,  equations,  algorithms, 
and  code.  Documentation  must  also  include  the  criteria  for  acceptable 
T^erformance  and  methods  for  test  and  evaluation.  Further,  documentation 
must  provide  for  easy  traceability,  trackability  and  accountability.  The 
type  of  documentation  that  sets  forth  this  information  varies  widely. 

For  ECS  computer  programs,  it  normally  is  contained  in  a number  of  cocu- 
ments,  as  summarized  in  Section  VI,  or  it  may  be  contained  in  a single 
document.  For  example,  many  of  the  F-111  OFF  support  software  programs 
are  documented  with  a single  combined  functional  and  implementation  speci- 
fication which  is  embedded  in  the  computer  program  code  and  stored  on  the 

1.  The  emphasis  of  this  Section  is  on  configuration  management  as  it 
applies  to  software.  It  is  assumed  that  the  hardware  support  elements 
are  manar-ed  in  accordance  with  standard  configuration  management  principles 
as  they  apply  to  an  engineering  laboratory  environment. 
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laboratory  host  computer  system.  Discretion  must  be  used  in  tailoring 
documentation  to  the  specific  application.  For  example,  documentation 
required  for  an  off-the-shelf  compiler  vihich  is  envisioned  to  change  very 
infrequently  will  be  less  than  that  required  for  a data  reduction  program 
which  is  envisioned  to  change  occasionally;  and  documentation  for  the 
data  reduction  program  will  be  much  less  than  that  required  for  an  SGS 
computer  program  which  is  envisioned  to  change  frequently.  Normally, 
documentation  will  consist  of  a requirements  document,  design  and  product 
specifications,  program  ].istings,  object  and  source  code,  test  documents, 
version  description  documents;  and  control  and  status  accounting  records, 
i.e.,  configuration  indexes,  change  requests,  change  proposals,  and 
change  logs. 

It  is  extremely  important  to  carefully  scrutinize  and  tailor  all 
documentation  to  fit  the  specific  computer  program  application  and  support 
requirement.  Further,  every  effort  should  be  made  to  standardize  docu- 
mentation and  to  develop  formats  that  are  adaptable  to  automated  updating, 
control  and  status  accounting.  Documentation  tends  to  be  voluminous, 
extremely  expensive  and  time  consuming  to  maintain,  and  is  worthless 
if  out  of  date.  The  weapon  system  flexibility  provided  by  the  ECS  soft- 
ware can  be  lost  through  indiscriminate  documentation  requirements. 
Voluminous  documentation  cannot  possibly  be  updated  at  the  pace  at  which 
software  is  capable  of  being  channed,  causing  one  of  two  things  to  happen: 
(l)  either  the  software  change  pace  is  slowed  down,  making  the  support 
system  non-responsive  to  the  user's  needs;  or  (2)  the  docmentation  does 
not  get  updated  and  eventually  brink's  the  whole  operation  to  a stop. 

This  can  be  avoided  by  usinr  prudent  judgment  in  tailoring  documentation 
to  the  need. 
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Another  important  aspect  of  configiiration  identification  is  the  number 
and  marking  of  computer  programs  and  documentation.  In  order  to  track  com- 
puter program  baselines  and  changes,  a computer  program  numbering  and 
marking  system  must  be  implemented  which  is  formalized  and  standardized, 
(Baseline  is  defined  as  "a  configuration  identification  document  or  a set 
of  such  documents  formally  designated  and  fixed  at  a specific  time  during 
a Cl’s  life  cycle.  Baselines,  plus  approved  changes  from  those  baselines, 
constitute  the  current  configuration  identification",)  (6:Encl  l,l) 

The  system  should  identify  all  computer  programs  with  a distinguishing 
baseline  part  number  and  subsequent  changes  with  version  description  numbers. 
Documentation  should  track  with  a basic  document  number  and  subsequent  re- 
vision numbers.  The  system  must  provide  a one-to-one  correlation  between 
computer  programs  and  documentation;  must  provide  traceability  to  the 
original  baseline;  and  facilitate  control  and  status  accounting.  (1:6), 

(5),  (6),  (10),  (12),  (17) 

G0?IFIGURATI0N  CONTROL; 

Configuration  control  is  defined  as  "the  systematic  evaluation,  coor- 
dination, approval  or  disapproval,  and  implementation  of  all  approved 
changes  in  the  configuration  of  a Cl  after  formal  establishment  of  its 
configuration  identification".  (5^2)  Its  primary  purpose  in  the  ECS 
software  support  environment  is  to  control  changes  to  the  software.  In 
addition,  it  controls  documentation,  controls  the  software  change  release, 
and  assures  that  a systematic  development  process  is  followed  which  results 
in  a released  computer  program  whose  configuration  is  accurately  reflected 
in  the  documentation.  Configuration  control  is  accomplished  through  a 
configuration  control  board  (CC3),  and  a series  of  reviews  and  audits. 
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Ficiire  IV-1  illustrates  a software  chanr^e  control  process  that  is 
suggested  for  ICGS  software  support.  It  tracks  closely  with  the  guidance 
promulgated  by  AFR  800-14  and  is  similar  to  the  change  control  process  in 
use  for  F-111  OFP's  and  also  similar  to  the  one  being  implemented  for  F-15, 
F-l6,  F-4E,  and  PAVE  TACK  OFP's.  Basic  differences  between  this  process 
and  the  standard  configuration  control  process  are:  (l)  a computer  prognram 

configuration  sub-board  (CPGSB)  subservient  to  the  weapon  system  GGB  ap- 
proves or  disapproves  changes;  and  (2)  software  changes  are  not  segregated 
in  accordance  with  the  MIL-STD  480  definition  of  Glass  I and  Glass  II 
changes,  but  are  segregated  according  to  whether  they  are  software  changes 
only  (no  weapon  system  ha,rdware  impacts)  and  according  to  whether  they  can 
be  implemented  within  the  existing  sustained  level  of  effort  support. 

All  proposed  EGS  software  changes  follow  this  process.  An  abbreviated 
form  of  this  process  is  used  for  changes  to  support  software. 

Ghange  requests  are  initiated  by  submitting  a Ghange  Request  Form 
of  the  type  shown  in  Appendix  A.  This  form  is  routed  through  appropriate 
channels  and  eventually  to  the  orranization  responsible  for  EGS  software 
maintenance,  where  it  is  logged,  documented  and  analyzed.  The  requested 
change  can  range  from  a correction  to  a system  anomaly,  perceived  as  a 
software  problem,  to  the  addition  of  a new  capability,  perceived  as  having 
a software  solution.  The  analysis,  as  outlined  in  Section  III,  investi- 
gates all  aspects  and  impacts  of  the  change  and  scopes  the  development 
and  implementation  effort.  Interfaces  are  one  of  the  most  important  areas 
and  must  be  investigated  thoroughly,  as  something  inadvertently  overlooked 
can  cause  serious  cost,  schedule,  and  performance  problems  downstream. 

For  example,  a change  made  to  the  F-111  OFP's  resulted  in  a compatibility 
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Computer  Program  Change  Proposal 
Computer  Program  Configuration  Suh-Board 


problem  with  the  aircraft  cockpit  panels  resulting'  in  a 300  thousand  dollar 
panel  mod.  The  more  common  interfaces  v.’ith  the  ECS  computer  programs  which 
warrant  close  scrutiny  are: 

• User/operator 

• ECS  computers 

• Weapon  system  equipment 

• Trainers  and  mission  simulators 

• Weapon  system  test  and  support  eo^uipment,  e.g.  , 
automatic  test  equipment  for  the  computers 

• Technical  data  system 

• Documentation 

• ECS  software  support  system 

The  analysis  results  in  one  of  three  decisions:  (l)  the  software 

change  is  feasible  and  has  no  vjeapon  system  hardware  impact;  (2)  the  change 
is  infeasible,  (e.g.,  does  not  provide  the  results  the  requestor  envisioned, 
violates  weapon  system  capability  or  enrineering  principles,  is  beyond  the 
computer  capability,  is  beyond  the  capability  of  the  sustaininr  level  of 
effort,  is  excessively  risky  in  terms  of  cost,  schedule  and/or  performance); 
and  (3)  the  change  impacts  weapon  system  hartware.  In  this  case,  the  change 
request  is  routed  to  the  weapon  system  manarer  for  }>rocessing  as  a hardware 
engineering  change  proT-Josa,!  (EG?)  or  other  appropriate  action. 

Feasible  software  chan- es  are  accumulated  and  periodically  reviewed 
by  the  user  and  supporting  a/roncy  to  determine  whether  they  should  be 
incorporated  into  an  EGG  computer  pror'ram  block  change.  Those  charges 
which  arc  determined  to  bo  of  too  low  a priority  to  commit  resources  are 
deferred.  Those  which  do  vjarrant  implementation  are  defined  in  a block 
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chanpe  requirements  dociunent,  and  a computer  proftram  chang'e  proposal  (GPGP) 
is  nrepared  and  subnitted  to  the  GPGSB,  which  has  the  delepated  authority 
to  approve  or  disapprove  software  changes.  The  GPGSB  is  made  up  of  members 
from  both  the  supportimr  and  usinp  apency  and  is  the  formal  controllinf' 
point  for  software  chanpes  and  the  intcp;ration  of  those  changes  with  the 
weapon  system.  They  review  the  validity  of  requirements  and  review  pro- 
posed solutions  to  assure  that  they  are  feasible,  axiequately  defined,  do 
not  represent  high  risk,  are  compatible  with  the  weapon  system  and  can  be 
accomplished  vathin  existing  resources. 

Approved  changes  process  through  final  development,  test  and  docu- 
mentation which  is  an  iterative  process  controlled  by  a series  of  reviews 
and  audits.  Ghanges  that  evolve  during  development  and  test  which  impact 
the  approved  baseline  are  processed  back  through  the  GPGSB  for  approval. 

The  final  documented  EGS  computer  program,  after  acceptance  by  the  user, 
is  released  for  field  implementation  through  a formal  and  controlled  re- 
lease system.  This  system  assures  that  the  EGS  computer  program  has  satis- 
factorily passed  the  functional  and  physical  configuration  audits  and  is 
compatible  and  complies  v;ith  the  weapon  system  change  release  system. 

In  summary,  this  change  control  process;  (l)  controls  the  approval 
and  implementation  of  block  changes  to  EGS  computer  urograms;  (2)  elimin- 
ates softv;are  changes  from  getting  bogged  dovm  in  the  full  GGB  process 
or  from  competing  with  hardware  changes  for  priorities;  and  (3)  nermits 
the  chan'^e  approval/disapproval  decision  to  be  made  by  people  who  are 
familiar  with  the  EGS  computer  programs  and  support  system.  This  results 
in  a more  efficient  and  responsive  software  change  process.  (1:6),  (5). 
(6),  (10),  (14),  (12),  (17) 


1,  A GPGP  is  an  EGP  tailored  to  software  application  . 
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, Another  important  function  of  confi(;>;uration  management  is  documentation 

f 

! control.  This  function  assures  that  all  documentation  used  for  computer 

I 

] program  configuration  identification  and  copies  of  all  computer  programs 

• are  controlled  and  stored  in  a safe  and  secure  area.  Lisually,  this  entails 

j establishment  of  a master  library  and  a worklnr  library.  The  master  library 

( retains  the  master  copy,  usually  on  microfilm  or  microfiche,  of  all  docu- 

I mentation  and  computer  programs.  The  vjorkinr  library  contains  copies  of 

I current  master  documentation.  These  documents  arc  required  and  used  by 

personnel  to  support  software  maintenance  and  modifications.  It  is  the 
responsibility  of  configuration  management  to:  (l)  assure  that  all  docu- 

mentation and  comnuter  programs  in  these  libraries  are  strictly  controlled, 
remain  current,  and  reflect  the  exact  configurations  of  the  ECS  software 
and  support  systems;  (2)  control  unauthorized  chan{'es  to  documentation 
and  softvjare;  and  (3)  control  the  use  of  unauthorized  documentation  and 
software  in  support  of  the  EGG  software  and  support  system. 

GOITIGURATION  STATUS  AGGOUNTING; 

Gonfiguration  Status  Accounting  is  defined  as  "the  recording  and 
reporting  of  the  information  that  is  needed  to  manage  configuration  effec- 
tively, including  a listing  of  the  an-Droved  configuration  identification, 
the  status  of  proposed  changes  to  confiriiration , and  implementation  status 
of  approved  chan."es".  (5s2)  Its  nurpose  in  the  EGS  software  support  en- 
vironment is  to:  (l)  provide  management  with  feedback  information  to 

monitor  Implementation  of  software  changes;  (2)  identify  configuration 
mana'~ement  problems  in  a timely  manner;  (3)  provide  historical  and  current 
information  on  the  EGS  software  and  support  system;  (4)  monitor  and  rerort 
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on  the  state  of  the  software  and  software  documentation;  and  (5)  standard- 
ize and  eliminate  redundant  documentation.  This  is  accomplished  by  parti- 
cipation in  technical  reviews,  conductinfr  configuration  audits  and  makia':; 
informal  inspections  of  software  and  docmentation. 

Computer  pro, 'crams  and  documentation  status  is  maintained  in  a con- 
figuration index  and  a change  status  report.  The  configuration  index  is 
a log  which  provides  a running  and  current  status  of  all  computer  programs 
and  documentation.  The  computer  program  change  status  report,  which  is 
normally  a supplement  to  the  configuration  index,  is  a log  which  provides 
a historical  and  current  running  account  on  the  status  of  all  computer 
program  changes,  ilormally,  there  are  senarate  logs  for  each  ECS  software 
configuration  and  another  for  the  support  software.  In  practice,  these 
logs  should  be  automated  to  facilitate  compiling  data.  They  provide  manage- 
ment with  up-to-date  visibility  a,nd  status  of  the  software  and  documenta- 
tion configuration. 

During  technical  reviews,  it  is  the  responsibility  of  configuration  ^ 

management  to:  (l)  assess  the  software  configuration  identification  and  I 

I 

change  control;  (2)  assure  that  the  computer  program  design  is  being  ade- 
quately documented,  in  accordance  with  established  configuration  management  ■ 

practices;  and  (3)  document  and  renort  any  deficiencies. 

At  the  end  of  the  softv:are  change  development  cycle,  it  is  the  re- 
snonsibility  of  configuration  management  to  conduct  two  audits,  the  func- 
tional configuration  audit  and  the  physical  configuration  audit.  These 
audits  must  be  satisfactorily  completed  before  the  software  is  released 
for  operational  use. 

The  functional  configuration  audit  reviews  tost  results  and  compares 
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them  against  functional  performance  requirements  contained  in  computer 
program  specifications.  It  assures  that  these  req.uirements  were  used  as 
the  criteria  against  which  computer  programs  were  tested  and  that  results 
reflect  that  the  criteria  were  oa-tisfied.  All  discrepancies  are  reported 
so  that  corrective  action  can  be  initiated.  The  test  results  become  a 
part  of  the  configuration  identification  and  are  incorporated  into  the 
version  description  document. 

The  physical  configuration  audit  reviews  the  final  computer  program 
confip-uration  and  compares  it  against  documentation  to  assure  that  docu- 
mentation reflects  the  as-built  computer  program.  To  do  this,  a listing 
of  the  computer  program  code  is  generated  and  dociomentation  is  reviewed 
to  assure  that  it  accurately,  thoroughly  and  clearly  tracks  the  computer 
program  from  the  requirements  through  the  code.  Documentation  is  also 
reviewed  for  completeness  and  conformity  to  established  standards.  Dis- 
crepancies are  reported  so  that  corrective  action  can  be  initiated.  The 
documentation  becomes  the  configuration  identification  for  the  newly  estab- 
lished confi'^xiration  baseline  of  the  computer  program  GI.  (5).  (6),  (lO), 
(12),  (17) 

AUTOMTED  DOCUKBNTATION  GENERATION; 

Even  with  a major  effort  to  tailor  software  documentation,  the  amount 
can  still  be  awesome,  requiring  considerable  resources  to  maintain,  contiml 
and  update.  It  is  not  uncommon  to  find  that  20  - 30  percent  of  the  total 
time  spent  in  maintaining  and  modifying  software  is  consumed  in  documenta- 
tion tasks.  One  way  to  alleviate  some  of  the  burden  is  to  automate  docu- 
mentation generation.  This,  in  itself,  creates  more  computer  programs  and 
requirements  for  more  documentation,  but  can  have  big  pay-offs;  and  might 
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be  the  difference  between  a responsive  and  non-responsive  ECS  software 
support  system. 

The  Sacramento  Air  Logistics  Center,  in  an  effort  to  keep  up  with 
their  ECS  software  change  commitment  for  F-111  OFP's,  found  it  necessary 
to  implement  automated  documentation  generation.  The  system  developed 
does  not  generate  all  documentation,  as  it  is  not  fully  implemented,  but 
greatly  reduces  the  amount  of  manual  tasks  that  were  previously  required. 
Essentially,  the  system,  operating  on  the  laboratory  host  computer,  sorts, 
manipulates  and  generates  all  data  pertinent  to  ECS  software  chanre  re- 
quirements. This  data  is  compiled  and  used  to  generate  the  block  change 
requirements  document,  design  specification,  CPCP,  test  plan  and  procedures, 
charge  narratives,  status  and  code.  During  development,  this  data  is 
automatically  updated  and  correlated  with  the  generation  of  the  block 
change  development,  engineering  and  release  baselines.  This  process  is 
illustrated  in  Figiare  IV-2.  (l7sl^-3,9) 

Requirements  for  automated  documentation  should  be  stated,  and 
spelled  out,  in  the  planning  docment,  e.g,,  CRISP.  This  will  enable 
the  acquisition  contractor  to  develop  documentation  that  is  computer 
generated.  It  is  very  difficult,  time  consuming,  and  expensive  to  con- 
vert a manual  system  to  an  automated  system,  as  is  the  case  for  the  F-111 
pro'^rams. 

In  summary,  the  benefits  derived  from  sound  software  configuration 
nana^'ement  are; 

• Well  defined  and  documented  SG3  software  and 
software  support  system 

• Efficient,  effective,  and  responsive  change 
control 
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• Total  system  traceability 


• Software  delivered  in  accordance  with  users 
requirements 

• Delivered  software  accurately  and  completely 
described  by  standard  documentation 

• Documentation  tailored  to  the  specific  need 

• Total  software  visibility 


SECTION  V 


TESTING 

Testing  is  perhaps  the  most  important  aspect  of  the  softviare  develop- 
ment process,  as  it  ultimately  demonstrates  and  verifies  the  success  or 
failure  of  the  development  program.  Testing  is,  however,  expensive.  His- 
torical data  compiled  on  several  major  development  programs  Indicates  that 
45-50  percent  of  development  efforts  are  expended  in  some  form  of  testing. 
(2:52)  In  addition  to  the  testing  effort,  major  costs  are  also  invested 
in  test  resources.  For  example,  to  provide  on-going  engineering  support 
for  the  F-111  OFP's,  and  for  avionics  system  integration,  a 20  million 
dollar  Investment  in  laboratory  resources  has  been  mad.e,  and  two  F-111 
aircraft  have  been  permanently  assigned  to  this  effort.  The  overriding 
requirement  for  these  resources  is  OFF  testing.  These  resources  are  illus- 
trated in  Figure  V-1.  (15) 

Figure  V-2  illustrates  the  challenge  of  software  testing.  It  can  be 
concluded  from  this  figure  that  it  is  literally  impossible  to  prove,  through 
testing  or  by  any  other  means,  that  even  the  simplest  computer  program  or 
program  change  is  completely  free  of  eirror.  In  fact,  the  Dijkstra  meixim 
states  "Testing  shows  the  presence,  not  the  absence  of  bugs".  (3*7)  Test- 
ing, therefore,  finds  errors  and  the  more  testing  that  is  accomplished,  the 
higher  the  probability  that  the  software  is  free  of  errors  and  satisfies 
performance,  i.e.,  the  greater  the  software  reliability^.  This  is  illus- 
trated in  Figure  V-3.  However,  prudent  judgment  must  be  exercised  in  deter- 
mining the  amount  of  testing  by  comparing  cost  expended  to  benefit  gained. 

1.  Software  reliability  is  generally  stated  as  the  probability  that  a 
computer  program  will  perform  its  intended  function  for  a stated  period 
of  time  in  a specified  environment. 
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AVIONICS  INTEGRATION  AREA 


(15:245) 


TESTING  IS  MOST  EXPENSIVE  PART  OF  DEVELOPMENT 
TESTING  IS  AT  BEST  "SELECTIVE" 


:FIjTSR  SOFTi'JARS  RELIABILITY 


In  spite  of  this  rather  bleak  picture,  softvfare  testing  can  be  cost 
effective  in  producing  software  with  a high  degree  of  confidence  in  its 
reliability.  To  accomplish  this  requires  detailed  planning,  explicit 
definition  of  test  objectives  and  a systematic  approach  in  implementing 
objectives.  This  section  addresses  test  objectives,  planning,  procedures, 
and  methods  as  they  apply  to  ECS  software  support. 

TEST  OBJECTIVES; 

Testing  can  be  separated  into  two  categories,  development  test  and 
evaluation  (DT&S)  and  operational  test  and  evaluation  (0T£:S). 

DTd-E,  as  it  applies  to  ECS  softvfare  changes,  starts  during  the 
initial  analysis  of  change  requirements  and  continues  throughout  the 
development  cycle.  DT&S  is  accomplished  by  the  agency  having  responsi- 
bility for  developing  the  ECS  software  changes  and  is  monitored  by  the 
user.  DT&E  objectives  are:  (l)  to  evaluate  the  technology,  design  and 

engineering  of  softvfare  changes;  (2)  to  demonstrate  that  risks  to  the 
weapon  system,  if  any,  have  been  eliminated;  (3)  to  demonstrate  that 
softvfare  changes  meet  the  specification  requirements;  and  (4)  to  demon- 
strate that  the  development  process  is  complete.  (4) 

OT&E  starts  vfhen  the  software  changes  are  turned  over  to  the  user 
for  test  and  evaluation  and  is  completed  prior  to  the  user  acceptance. 

For  example,  in  the  case  of  F-111  OFP's,  OTiiE  commences  shortly  after  j 

the  start  of  DT£;E  flight  testing  and  is  conducted  in  parallel  ifith  it,  j 

as  shovm  in  Section  III,  Figure  4.  OT&E  is  performed  by  the  user  and  [ '■ 

its  objective  is  to  demonstrate  that  the  software  changes  perform  their  \ 

intended  functions  in  a user  mission  environment.  (4)  ! 1 

Since  it  is  more  effective  to  test  software  for  the  presence  of  errors,  4 


then  test  objectives  should  be  directed  at  the  most  probable  error  sources. 

Lt.  Gol.  Manley,  in  his  article  "Embedded  Computer  System  Software  Reliability" 
(11:14)  summarizes  a number  of  these  error  sources.  They  are  divided  into 
two  general  categories,  design  and  coding  errors  and  externally  caused  failures. 

Design  and  coding  errors  are  those  errors  caused  by  personnel  during 
development.  They  include: 

"Syntax  Errors:  The  syntax  of  a computer  language  is  the  precise 

definition  of  words,  statements  and  combinations  thereof  that  a compiler 
(or  language  traJislator)  will  accept.  Syntax  errors  usually  result  from 
incorrectly  matching  programming  languaiges  with  compilers.  Fortunately, 
most  syntax  errors  can  be  detected  by  good  compilers  and  correction  is  not 
difficult. 

Semantics  Problems:  In  translating  user  requirements  into  pro- 

gramming languages  and  programming  languages  into  machine  code,  misinter- 
pretations can  occur.  This  is  one  of  the  more  difficult  sources  of  soft- 
ware errors  to  detect  and  correct. 

Logic  Errors;  Caused  during  design  and  coding,  these  errors 
result  in  computer  program  logic  deficiencies,  such  as  the  creation  of 
continuous  loops  and  impossible  states  that  end  in  deadlocks. 

Algorithmic  Errors:  Many  software  failures  are  caused  by  mathe- 

matical errors  such  as  incorrect  scaling  of  variables.  Iterative  rounding 
errors  can  reduce  the  precision  of  variables  below  acceptable  limits.  Divi- 
sion by  zero,  multiplication  of  excessively  large  numbers,  and  similar  mathe- 
matical operations  can  cause  overflow  problems  and  subsequent  incorrect 
computer  outputs" . ( 1 1 : 14 ) 

Externally  caused  failures  are  external  environment  influences  which 
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can  cause  the  software  to  fail  even  though  it  is  logically  correct.  They 
include : 

"Computer  Hardware  Failures;  Certain  failure  modes  of  the  com- 
puter itself  are  a source  of  software  malfunctions  not  easily  traceable 
to  the  hardware.  Examples  are:  imperfections  on  maignetic  siarfaces,  random 

power  fluctuations  and  the  malfunction  of  single-core  elements,  or  flip- 
flops. 

Interactions  With  Other  System  Components:  Software  logic  that 

requires  continuous  interaction  with  other  system  components,  such  as  servo- 
mechanisms, inertial  navigators,  and  radar  sets,  can  fail  due  to  the  transient 
malfunctions  in  the  electromechanical  components.  Since  software  operates 
on  well-defined  computer  data  inputs,  modifications  to  those  inputs  can 
cause  abnormal  logical  operations. 

Incorrect  Human  Inputs:  Human  inputs  through  button  pushing  or 

keying  can  generate  erroneous  computer  data  inputs".  (11:14) 

Environmental  Changes:  An  embedded  computer  system  can  be  sub- 

jected to  temperature,  pressure,  humidity  and  electro-magnetic  radiation 
conditions  which  can  cause  hardware  malfunctions  and  subsequent  software 
failures. 

Interface  Incompatibilities:  Changes  and/or  degradation  in 

hardware  performance  can  result  in  changes  in  computer  interface  timing 
and  signal  characteristics  resulting  in  compatibility  problems  which  can 
cause  software  failures. 

TEST  PLANNING  AND  PROCEDURES: 

The  best  of  intentions  generally  never  get  implemented  without  ade- 
quate planning- -so  goes  it  with  software  testing.  Managers  typically  look 


very  optimistically  at  test  objectives  and  tend  to  gloss  over  required 
resources  for  implementation,  resulting  in  gross  underestimates  of  the 
test  program.  The  solution  to  this  lies  in  early  detailed  planning. 

The  software  test  plan  must  be  prepared  early  in  change  development 
and  is  updated  as  more  detailed  information  becomes  available.  It  describes 
in  detail  test  objectives,  course  of  action  for  accomplishing  objectives, 
and  defines  criteria  and  requirements  against  which  the  software  v;ill  be 
tested  to  demonstrate  that  it  satisfies  objectives.  The  plan  describes 

i the  software  to  be  tested  and  defines  responsible  test  agencies  and  their 

roles;  test  locations;  test  environment;  schedule;  limitations  and/or 
constraints;  data  acquisition,  reduction  and  analysis  requirements;  test 

I 

1 evaluation  criteria;  and  resource  requirements,  such  as  facilities,  labora- 

j tory  and  weapon  system  equipment,  support  software  and  personnel.  (20) 

1 

I Test  procedures  implement  objectives  and  requirements  of  the  test 

i plan.  They  are  prepared  for  each  level  of  test;  describe  the  unit  under 

I 

I test;  outline  the  method  and  control  of  test;  prescribe  methods  for  record- 

l ing,  evaluating  and  reporting  test  results;  and  specify  test  resources. 

The  test  report  documents  and  presents  test  results,  conclusions, 
and  recommendations.  It  summarizes  test  results  in  terras  of  satisfying 
test  objectives;  and  discusses  test  anomalies  and  problems.  It,  along 
with  data  recorded  in  test  procedures,  provides  the  basis  for  the  func- 
tional configuration  audit  and  acceptance  and  release  of  software  changes 
for  operational  implementation.  (2l) 

TIST  METHODS; 

The  most  widely  used  test  method  for  SGS  software  is  what  is  commonly 
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termed  "verification,  validation  ajid  certification".  Essentially,  it  is  a 


systematic  methodology  employed  to  assure  that  each  phase  of  the  development 
process  is  sequentially  tested  in  accordance  with  specified  procedures  and 
against  stated  criteria.  It  is  an  iterative,  regressive  process  which  stsirts 
by  verifying  ECS  software  change  requirements  and  is  completed  with  certifi- 
cation of  the  software  in  a mission  environment. 

Verification,  validation  and  certification  are  either  implemented  by 
the  organization  developing  the  change  or  by  an  independent  group  or  agency. 
The  advantage  of  using  an  independent  source  is  an  unbiased  assessment. 
Further,  they  are  more  prone  to  look  for  failure  modes  in  lieu  of  verifying 
acceptable  performance.  The  main  disadvantage  is  resources.  There  obviously 
is  a certain  amount  of  redundancy  which  generates  a requirement  for  additional 
people  and  a queuing  of  resources.  The  personnel  requirement  can  be  appre- 
ciable since  these  people  must  be  on  board  during  the  complete  development 
cycle.  For  exaunple,  to  implement  independent  verification  and  validation 
for  the  F-111  OFP's  is  requiring  approximately  a 15  percent  increase  in 
support  personnel.  (l8) 

Verification  is  iterative  and  assures  that  each  step  in  the  develop- 
ment process  is  correct.  In  a step-by-step  sequence  through  each  phase 
of  development,  it  checks  documentation  and  tests  outputs  against  input 
criteria.  It  assures  that  system  requirements  have  been  properly  translated 
into  ECS  software  requirements  and  that  ECS  software  specifications  have 
been  correctly  derived  from  these  requirements.  It  assures  that  the  pro- 
gram is  free  from  coding  or  structural  errors  and  that  the  algorithms  and 
equations  used  to  Implement  requirements  are  operationally  correct  and 
produce  expected  results.  Verification  is  normally  done  in  the  laboratory 
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using  the  host  computer  system,  dynamic  simulator  and  verification  software 
tools.  (These  resources  are  discussed  in  Section  VI  and  Appendix  B. ) (9) 

Validation  tests  the  ECS  computer  program  vfhile  performing  in  the 
ECS  computer.  It  validates  performance  against  the  system  specification. 
For  example,  in  the  case  of  F-111  OFP's,  tests  are  conducted  in  the  F-111 
Avionics  Integration  Support  Facility,  using  the  dynamic  simulator  and 
system  mock-ups.  On  the  dynamic  simulator,  output  parameters  such  as 
range  to  target,  destination  coordinates  and  present  position  are  verified 
against  simulated  input  scenarios:  and  on  the  mock-ups,  system  interfaces, 
compatibility,  signal  characteristics,  and  timing  are  validated. 

Certification  tests  the  ECS  computer  program  in  its  actual  mission 
environment,  assuring  that  mission  requirements,  system  effectiveness, 
operational  availability,  dependability  and  capability  are  satisfied. 
Certification  is  always  performed  with  actual  mission  equipment  which  is 
modified  only  to  the  extent  necessary  for  data  acquisition.  For  example, 
aircraft  OFP’s  are  always  certified  using  operational  aircraft  modified 
only  to  accommodate  instrumentation  and  data  acquisition  systems.  (9) 

Figure  V-4  illustrates  the  verification,  validation,  certification 
process,  and  Figure  V-5  correlates  the  process  with  the  software  change 
development  cycle  and  test  resources. 

In  summary,  this  Section  has  highlighted  the  need  for  effective 
testing  of  ECS  softviare  changes  and  presented  a systematic  test  approach. 
This  approach  stressed  early  and  thorough  planning,  explicit  definition 
of  test  objectives,  detailed  test  procedures  and  reports  and  the  verifi- 
cation, validation,  certification  method  of  implementation. 
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Firure  V-5 
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RESOURCES 


As  noted  in  previous  sections,  the  task  of  supporting  ECS  software 
is  extensive  with  software  modifications  viewed  as  a continuous  series  of 
microcosms  of  the  acquisition  development  cycle.  This  section  summarizes 
resources  required  to  provide  efficient,  effective  and  responsive  ECS 
software  support;  and  addresses  considerations  that  should  be  given  and 
trade-offs  that  should  be  made  in  their  selection.  Resources  are  expen- 
sive and  require  considerable  lead  time  for  acquisition,  therefore,  dis- 
cretion must  be  used  in  establishing  resource  requirements  and  these 
requirements  must  be  known  early.  Resources  fall  into  five  major  cate- 
gories; personnel,  equipment,  software,  documentation,  and  facilities. 

Each  of  these  categories  are  briefly  discussed  in  this  section,  with  more 
detailed  information  provided  in  Appendix  B. 

Responsibility  for  providing  resources  and  determining  quantities, 
time  required  and  eventual  location  should  be  spelled  out  in  the  planning 
document.  In  the  case  of  the  Air  Force,  this  document  is  the  CRISP  (refer- 
ence Section  II ).  It  should  spell  out  the  support  approach  and  extensively 
detail  resource  requirements  to  include  funding,  acquisition  and  implementa- 
tion, allowing  sufficient  lead  time  so  that  all  resources  are  in  place 
and  operational  at  the  time  of  system  responsibility  transfer  from  the 
developing  to  the  supporting  agency. 

As  mentioned,  resources  are  expensive  and  every  effort  should  be 
expended  to  minimize  the  cost.  To  this  end,  several  significant  things 
car.  be  done. 
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First,  and  ■nerhaps  most  ir.rortar.t,  is  to  finalize  the  decision  on 
surr>ort  reauirenents , a.pency,  and  s’te  early  in  acq_uisition  development. 

If  this  is  done,  the  information  can  be  very  valua.ble  in  directinp  the 
contractor's  s.cquisition  development  so  t!';at  the  SGS  softv.'are  and  attendant 
sup'rortir.f;  resources  are  desi;;ned  vfith  me-ximum  maintainability  and  tra.ns- 
ferabil-ity  in  mind.  Softvraro  can  be  developed  usin'"  hi"h  order  lanruaye 
(HOL),  structured  prorramr.inp: , modular  desiyn  and  top  down  rrofpramriiiip; 
techniques  resulting  in  ea,sicr  nainter.ance  a,nd  fe'.?er  support  personnel 
requirements.  Engineering  and  rropremning  documentation  used  by  the  con- 
tra.ctor  ca.r.  be  prepared  in  such  a v?ay  that  it  is  directly  tra.r.sf crable  to 
the  su-uTiorting  agency.  Support  software  can  be  developed  so  that  it  ic 
conratible  and  directly  transferable  to  suppor-tii.g  agencies  host  coneuters. 
Finally,  and  most  important,  system  integration  mock-ups,  dynamic  simulators, 
special  test  equipment,  in,T.trur.erta.tion,  and  data  acquisltioi  systems  used 
for  acquisition  devclorment  can  be  designed  ar.d  structured  so  that  they  are 
easily  relocatr.ble  to  the  supporting  site  after  syste."  deplo^vT.ent.  This 
early  ola-nninr  can  go  a long  way  in  negatin'”  the  requirement  for  t'.:o  sets 
of  resources,  one  for  acquisition  development  and  one  for  maintenance,  i.c., 
on-yoiny  EG3  software  support.  The  Air  Force,  for  l''-l6  CFi''s,  has  v.’orked 
out  this  type  of  an  arrangement  between  t'ne  developers  (Ajr  Force  Systems 
Gomjiand  and  Ge:'.f;ral  Dynamics),  and  the  sunporting  a.  ency  (Cgoei.  Air  Logis- 
tics Gentcr).  One  of  the  problems  vritli  th.is  tecanique  is  the  void  Ir 
s/jpport  tha.t  can  be  created  durinr  transition. 

T:ie  transition  problem  can  be  completely  eliminated  by  developin," 
the  softwa.ro  from  inception  a.t  the  desi "nated  supportinr  site.  In  this 
tyre  o:'  arraur'enont,  thorn  is  absolutely  no  duplication  of'  r 
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maximum  interchange  between  the  developing  and  supporting  agencies  prior 
to  transfer  and  little  risk  of  support  problems  during  transition  and 
transfer. 

Two  of  the  most  expensive  resources  required  for  ECS  software  manage 
ment  and  support  are  personnel  and  weapon  system  equipment.  Cost  of 
support  can  be  significantly  reduced  if  a support  site  is  selected  where 
the  bulk  of  these  resources  are  available  without  additional  acquisition. 
For  example,  the  F-111  OFP  support  is  located  at  the  Sacramento  Air  Logis 
tics  Center,  which  is  the  depot  for  the  F-111  aircraft  and  its  digital 
avionics.  To  support  F-111  OFP's  requires  approximately  60  technical  per 
sonnel,  the  use  of  some  ? - 10  million  dollars  worth  of  avionics  assets 
configured  into  laboratory  development  and  test  mock-ups,  and  the  use  of 
F-111  aircraft  for  flight  testing  OFP  changes.  The  same  system  equipment 
is  required  for  engineering  support  of  the  F-111  avionics  system.  Co- 
location  at  the  Sacramento  Air  Logistics  Center  permits  the  use  of  a 
single  laboratory  (The  F-111  Avionics  Integration  Support  Facility,  ref- 
erence Figure  V-l)  to  satisfy  both  requirements.  Go-location  further 
permits  the  laboratory  to  borrow  avionics  assets  from  the  depot,  elimi- 
nating the  need  for  procurring  dedicated  resources.  Also,  co-location 
at  the  Sacramento  Air  Logistics  Center  with  other  F-111  aircraft  engineer 
ing  disciplines  permits  sharing  of  flight  test  aircraft.  There  were  no 
appreciable  savings  in  personnel  costs  by  selecting  this  site  to  support 
F-111  OFP's  because  at  the  time  there  did  not  exist  a pool  of  software 
expertise.  However,  now  that  this  expertise  is  available,  it  becomes 
a factor  in  evaluating  support  alternatives  for  future  ECS  software 
workloads. 
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PERSOMEL; 


The  task  of  raana{:ement  and  support  of  ECS  software  is  closely  inte- 
grated with  the  weapon  system  management  and  is  highly  technical.  It 
requires  management  personnel  with  prograjn  management  and  technical  back- 
grounds, and  with  weapon  system  experience  and  expertise;  and  technical 
personnel  who  are  highly  skilled  in  system  engineering  and  software  dis- 
ciplines. These  personnel  Include  ECS  software  managers,  ECS  computer 
system  engineers,  software  engineers,  technical  analysts,  scientific  pro- 
grammers, data  processing  programmers,  test  engineers,  hardware  engineers, 
configuration  manaigers,  technicians,  technical  writers,  craftsmen,  clerical 
and  supply  personnel.  The  preponderance  of  personnel  req  red  to  support 
ECS  software  are  technical  and  the  number  depends  of  many  factors.  Some 
of  the  more  important  ones  are:  software  complexity,  saturation  of  the 

computers,  frequency  of  changes,  type  of  changes,  programming  language, 
software  structure,  quality  of  docimentation,  quality  of  development  and 
test  tools,  documentation,  test  and  integration  requirements,  routine 
maintenance  requirements,  and  the  experience  level  of  personnel.  All 
of  these  factors  must  be  considered  when  estimating  personnel  require- 
ments. Uo  attempt  is  made  in  this  report  to  tie  down  numbers,  but  only 
to  sxiggest  types  of  personnel  and  their  responsibilities  and  skills. 
Descriptions  of  support  personnel  are  summarized  in  Appendix  B.  More 
detailed  personnel  descriptions  are  available  in  the  reference.  (18) 

SQUirMENT; 

Equipment  constitutes  the  primary  ECS  software  development  and  test 
tools.  It  consists  of  both  laboratory  and  weapon  system  resources.  The 
sophistication  and  quantity  of  equipment  required  are  highly  dependent  on 
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the  Dartlciilar  ECS  software  beinp  supported  and  support  requirements.  How- 
ever, in  all  cases  some  type  of  host  computer,  dynamic  simulator,  equipment 
mock-up  ajad  acquisition  system  will  be  reouired,  as  well  as  some  weapon  system 
equipment . 

A host  computer  system  will  be  required  in  order  to;  (l)  compile  and 
reassemble  source  data,  (2)  renerate  protpram  code  and  documentation,  (3) 
operate  and  control  laboratory  simulators  and  mock-ups,  (^)  acquire,  reduce 
and  analyze  test  data,  (5)  store,  manipulate,  and  retrieve  data  and  docu- 
mentation, and  (6)  execute  miscellaneous  computer  programs. 

A dynamic  simulator  system  will  be  required  in  order  to  develop  and 
test  ECS  computer  programs  under  dynamic  conditions  in  a simulated  real 
world  environment. 

A weaoon  system  equipment  mock-up  will  be  required  in  order  to  test 
and  evaluate  interfaces  bet'ween  the  weapon  system  equipment  and  the  ECS 
software  vihich  includes  tests  for  timing,  compatibility,  environmental  con- 
ditions, RFl/SI'il,  etc. 

A data  acquisition  system  will  be  required  in  order  to  acquire  test 
data,  both  in  the  latoratory  arid  during  ECS  soft'.rarc  csrtificat j on., 

Weapon  system  equipment  will  be  required  in  order  to  support  ECS  soft- 
ware develooment,  and  to  sunport  laboratory  and  certification  tests. 

A more  detailed  description  of  this  equipment  is  provided  in  Appendix  B, 

30FT.VARE; 

Software  can  be  divided  into  two  categories,  ECS  computer  programs 
and  the  support  software  system.  ECS  computer  programs  are  sometimes  re- 
ferred to  as  the  end  item  computer  programs  or  operational  computer  programs. 
It  is  the  software  which  resides  in  the  weapon  system  computers,  e.g. , OFP's, 
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and  the  software  which  generates  the  requirement  for  a management  and 
support  system.  All  other  software  is  considered  in  the  support  category. 
The  sunport  software  system  consists  of  all  attendant  computer  programs 
used  in  the  nTOcess  of  effecting  changes  and  modifications  to  the  SGS  com- 
puter programs.  This  software  can  be  grouped  into  programming  aids,  verifi- 
cation tools,  host  computer  operating  systems,  utilities,  simulation  and 
test  bed  control  systems,  application  progreims,  display  programs,  data 
acquisition  software,  configuration  management  softviare,  and  miscellaneous 
programs.  The  total  number  of  programs  can  be  large  and  depends  on  the 
degree  of  laboratory  soohistication  and  automation.  For  example,  there 
are  some  ] 500  computer  programs  which  make  up  the  suoTJort  software  system 
for  seven  F-111  OFP's.  Descrintions  of  support  software  are  summarized  in 
Appendix  B. 

DOGUIiarTATION;^ 

Sufficient  documentation  is  essential  for  effective  configuration 
manarement  of  the  3GS  software  and  the  software  support  system.  Further, 
adequate  documentation  is  required  to  permit  ease  of  software  and  equipment 
maintenance  and  modification  by  competent  technical  personnel.  Two  things 
must  be  remembered  when  determining  documentation  requirements:  (l)  it  is 
expensive  to  procure  and  maintain;  and  {?.)  it  is  to  be  used  by  highly  com- 
petent technical  personnel  in  a laboratory  environment.  Therefore,  every 
effort  should  be  made  to  use  the  same  documentation  that  was  used  for 
acquisition  development,  and  commercial  documentation  should  be  acceptable 

1.  Docamentatlon  for  the  purpose  of  this  report  means  engineering  docu- 
mentation and  does  not  incDude  documentation  which  is  a part  of  the 
Technical  Data  Systems,  e.g. , technical  orders. 
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for  "off  the  shelf"  laboratory  softviare  and  equinment.  Documentation  can 
be  divided  into  three  categories:  SG3  software,  support  software  and 

hardware . 

SGS  Software;  Generally,  the  documentation  requirements  are  fairly 
straight  forward  for  the  SGS  computer  programs.  On  this  software,  the 
followinf'  documents  are  normally  specified:  (14:3-4) 

• ’./eapon  System  Specification 

• SGS  Gomputer  System  Specification 

• EGS  Software  Design  Specification 

• SGS  Soft'ware  Product  Specification 

• SGS  Software  Version  Description  Document 

• SGS  Software  Gonf iguration  Index 

• SGS  Software  Ghange  Status  Report 

• EGS  Software  Users  Manual 

• EGS  Software  Gomputer  Programming  Manual 

• SGS  Software  Test  Plans,  Procedures  and  Reports 

• EGS  Software  Program  Listings 

• EGS  Software  Source  and  Object  Programs 

• Interface  Gontrol  Documents 

It  is  generally  a good  practice  to  acquire  as  reference  material,  copies 
of  any  additional  data  or  documentation  that  the  contractor  used  in 
acquisition  development. 

Sunnort  Software;  Documentation  requirements  are  not  so  straight 
forward  for  the  support  software.  First,  the  support  software  may  or  may 
not  be  coming  in  total  from  the  acquisition  development  contractor  and 
second,  even  if  it  were,  it  (or  its  documentation)  may  not  have  been 
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specified  as  deliverable  on  the  acquisition  development  contract.  Further, 
it  is  very  impractical,  both  from  a cost  and  maintenance  aspect,  to  require 
all  of  the  documents  listed  above,  for  the  support  software.  The  important 
thing  here  is  to  ask  for  documentation  that  adequately  describes  each  support 
program  and  provides  sufficient  information  so  that  it  can  be  maintained  and 
changes  can  be  implemented.  Generally,  the  documentation  should,  as  a mini- 
mum, provide  narrative  descriptions,  baseline  descriptions  and  logic  flow; 
and  describe  each  program  in  terms  of  inputs,  outputs,  functions,  trans- 

i 

formations,  algorithms,  equations,  timing,  logic,  and  size.  This  information 
is  generally  contained  in  functional  and  imnlementation  specifications,  logic 
flow  diagrams,  program  listings,  source  and  object  programs,  programmer 
manuals,  and  user  manuals.  This  documentation  should  be  acceptable  in 
commerlcal  form. 

Hardware ; For  the  v/eapon  system  equipment,  specifications  and  drawings 
are  normally  delivered  on  the  acquisition  contract  and  it  should  only  be 
necessary  to  obtain  copies.  In  addition,  user  technical  data  will  normally 
be  available. 

For  other  laboratory  equipment,  commercial  documentation  should  be 
adequate.  The  important  thing  again  is  to  assure  that  the  documentation 
describes  system  performance,  and  provides  sufficient  detailed  description 
for  operation,  preventive  maintenance,  trouble  shooting,  repair,  part  re- 
placement and  modification.  The  documentation  should  include  performance 
specifications,  functional  descriptions,  theory  of  operations,  drawings, 
part  lists,  wiring  dia~rams,  user  manuals,  maintenance  procedures,  and 
Interface  drawings. 

! 
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I FACILITIES; 

Several  problems  generally  arise  concerning  facilities  required  to 
house  resources  previously  discussed.  The  most  important  is  the  lead 
time  for  procurement  if  they  are  not  available  or  require  major  modifi- 
cation. This  becomes  a pacing  item  because  of  military  construction  pro- 
curement  raimif ications.  Because  of  this,  it  is  absolutely  essential  that 
facilities  be  identified  early  to  avoid  having  equipment  and  no  place  to 
operate  it.  Since  most  of  the  equipment  Involves  computers  or  computer 
peripherals,  power  and  environmental  requirements  typically  are  the  sajue 
as  those  for  other  computer  operations,  i.e.,  stringent  power,  temperature, 
air  circulation,  and  dust  controls.  Normally,  these  facilities  also  re- 
quire controlled  access.  Further,  if  classified  data  will  be  processed, 
then  security  requirements  will  also  have  to  be  met.  All  of  these  re- 
quirements take  a great  deal  of  time  to  implement.  Therefore,  it  is 
always  advantageous  to  seek  out  facilities  which  already  meet  these 
requirements.  It  is  also  highly  desira.ble  to  have  facilities  that  have 
adequate  office  work  and  study  areas  for  support  personnel.  A great 
deal  of  efficiency  is  lost  if  personnel  are  not  co-located  with  the 
laboratory.  Floor  space  requirements  will  vary  with  system,  but  5 >000  - 
6,000  square  feet  is  not  an  unreasonable  requirement  for  the  averag;e 

system. 

I 

S’  In  summary,  this  section,  in  conjunction  with  Appendix  B,  has 

discussed  personnel,  equipment,  software,  documentation  and  facilities 
[ essential  for  ECS  software  support.  Emphasized  were  resource  cost 

[ (in  particular  personnel  and  weapon  system  equipment),  need  for  highly 
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qualified  personnel,  tailoring  of  documentation  requirements,  and  the 
need  to  identify  early  in  weapon  system  acquisition  development  all 


resource  requirements. 


i 
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SECTION  VII 


SUPPORT  ORGANIZATION 

Support  agency  policies  and  regiolations  normally  promulgate  organization 
structure  and  responsibilities  which  will  vary  with  agency  mission  and  support 
role.  If  ECS  software  support  is  one  of  the  agencies  assigned  responsibili- 
ties, then  a substantive  software  organization  is  essential  to  effective, 
efficient  and  responsive  support. 

It  is  suggested  that  a software  support  organization  structured  and 
interfaced  as  illustrated  in  Figure  VII-1  can  optimize  these  support  features. 
It  places  total  software  support  responsibility  and  resources  vinder  a single 
manager,  which  maocimizes  the  proficiency  of  software  personnel  and  maiximizes 
the  sharing  of  resources.  It  also  minimizes  excessive  coordination  and  job 
redundancy.  Further,  the  organization  interfaces  at  a level  where  it  cam 
be  responsive  to  all  system  managers  and  is  visible  to  the  user  for  direct 
commmiication  and  coordination. 

The  organization  is  structured  so  that  program  management  can  be  applied 
to  software  modifications.  Under  this  concept,  the  management  section  has 
responsibility  for  ECS  software  management,  as  well  as  responsibility  for 
budgeting,  financial  matters,  contracts,  training  and  resource  management, 
and  has  a program  manager  assigned  to  the  ECS  software  for  each  weapon  system. 
The  program  manager  has  complete  responsibility  for  software  modifications 
and  plans,  schedules,  budgets,  coordinates  and  directs  all  resources  used 
in  accomplishing  modification  tasks.  He  also  has  responsibility  for  the 
coordination  and  Interface  with  all  organizations  and  agencies  external  to 
the  software  organization.  The  respective  ECS  software  support  sections 
have  responsibility  for  technical  software  support  and  system  integration. 
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ECS  SOFT./ARE  SUl'TOliT  0, 


and  provide  the  personnel  required  to  develop  software  modifications  and 
changes,  staff  problems,  conduct  studies,  etc.  The  configviration  management 
section  has  responsibility  for  all  software  configuration  management  and 
formal  documentation.  The  test  section  has  responsibility  for  all  formal 
and  independent  testing  to  include  preparation  of  formal  test  plans,  pro- 
cedures and  reports,  and  test  implementation.  The  laboratory  support  section 
has  responsibility  for  all  laboratory  support  resources  (both  hardware  and 
software)  and  responsibility  for  providing  laboratory  support  services  for 
all  ECS  software  systems. 
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SECTION  VIII 


SUI-aiARY.  CONCLUSIONS  AND  REG01#:ENDATI0NS 

S^TiARYi 

In  summary,  this  report  has  addressed  and  discussed  management  and 
support  of  ECS  software  after  weapon  system  deployment.  It  included  sup- 
port planning,  requirements  and  alternatives  with  emphasis  placed  on  tailor- 
ing support  to  generic  software  classes.  The  change  process  was  discussed 
v;ith  emphasis  on  the  block  change  concept  and  development  cycle.  Software 
configuration  management  was  discussed  vfith  emphasis  on  the  requirement 
to  manage  software  as  configuration  items;  the  need  to  tailor  configuration 
management  principles  to  support  system  requirements, (in  particular  docu- 
mentation) • change  control;  the  computer  program  configuration  sub-board; 
status  accounting  and  audits;  and  automated  documentation  generation. 
Softvrare  testing  v;as  discussed  vfith  emphasis  on  test  objectives;  software 
error  sources;  test  plans,  procedures,  and  reports;  and  the  verification, 
validation,  certification  method  of  testing.  Support  resources  were  dis- 
cussed in  detail  with  emphasis  on  cost,  alternatives  for  minimizing  cost, 
types  of  personnel  skills,  and  laboratory  and  weapon  system  equipment. 
Finally,  a support  organization  was  suggested  which  places  all  ECS  soft- 
vfare  support  resources  at  a base  under  a single  software  manager. 

GOI.'GLuSICNS 

ECS  software  can  provide  the  weapon  system  with  tremendous  flexi- 
bility in  responding  to  problems  and  requirements  because  of  the  relative 
ease  at  which  it  can  be  changed. 
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However,  to  provide  this  capability  during  the  operational  phase 
of  the  weapon  system  life  cycle  requires  an  efficient,  effective  and 


responsive  ECS  softvjare  management  and  support  system  and  a major  resource 
commitment.  In  order  to  accomplish  this,  the  system: 

(1)  must  be  planned  early  in  weapon  system  acquisition 
development  and  fully  coordinated  with  the  vjeapon 
system  integrated  logistics  support  plan; 

(2)  must  include  a staff  of  highly  qualified  and 
experienced  personnel  and  a maintenance  and 
modification  laboratory  equipped  with  a host 
computer  system,  system  equipment  mock-ups; 
dynamic  simulation,  data  acquisition  and 
weapon  system  equipment;  and 

(3)  must  provide  a systematic  method  for  developing 
ECS  softvjare  changes  which  employs  strong  program 
management,  configuration  mancigement,  test  manage- 
ment and  system  engineering. 

RSGOMI'lEimATIONS : 

1.  Greater  emphasis  be  placed  on  educating  mana.'rement  on  require- 
ments and  benefits  of  ECS  software  support. 

2.  Stronger  and  more  substantive  management  xx)licies  and  support 
be  provided  in  acquiring  resources,  in  particular  personnel,  and  imple- 
menting ECS  software  support  plans. 


I.  5 NUMoC  f? 


CCV.PUirR  PROGRAM  REQUEST 


1.  ini.E 

3.  CO'.rUTER  PROGRAM  IDENTIFICAl  lOM 


4.  D:-.S(.  R1PTI0IVPR£SENT  OPERATION: 


5.  DESIRED  OPERATION; 


6.  REASON  FOR  CHANGE: 


7.  CHANGE  HISTORY/R ELATED  CHANGES: 


8.  reque;sted  by: 

name  oi«gn 

THOME 

10.  REQUESTING  COMM/.ND; 

/ fproval 

N/vmE 

phone 

9.  REQUESTING  #’ING;  COORDINATION 


NA  WE 


THOME 


11.  SUPPORTING  AGENCY:  APPROVAL 


name 

A 2M- I- 


pmone 


APPEI'irilX  B 


SUMMARY  DESCRIPTIONS  OF  RESOURCES 
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PERSONNEL; 

ECS  Software  Mancigementt  These  personnel  maneige  the  overall  ECS  soft- 
ware support  effort  in  concert  with  the  weapon  system  support  program.  They 
plan  and  budget  for  the  development  and  implementation  of  supporting  resources, 

, forecast  personnel  requirements,  and  recruit  and  train  new  personnel.  They 

I 

establish  ECS  software  support  policy,  procedures,  and  methods;  allocate 
and  commit  resources;  provide  administrative  and  technical  guidance  to  sub- 
ordinates; and  assure  that  ECS  software  objectives  remain  in  consonance 
! with  the  weapon  system  support  objectives.  In  order  to  effectively  perform 

I 

' assigned  tasks,  managers  must  possess  a detailed  knowledge  of  the  weapon 

I 

system,  ECS  software,  and  ECS  software  support  resources.  Further,  they 

/ 

must  have  a thorovigh  knowledge  of  the  support  eigency  and  its  relationship 

with  other  DoD  organizations  and  be  capable  of  effectively  communicating 

and  coordinating  through  the  vertical  and  horizontal  management  structure. 

Embedded  Computer  System  Engineers;  These  personnel  perform  the  func- 

^ tion  of  program  manziger  for  system  design  and  development  of  large  ECS 

I /'  software  modifications,  i.e.,  block  changes.  They  establish  change  re- 

1. 

I quirements  through  coordination  with  the  users  and  system  manager;  direct 

I 

I the  preparation  of  change  specifications;  and  plan,  schedule  and  budget 

for  the  complete  modification  process  to  include  the  investigation  and 

■ 

analysis  of  requirements;  design,  development,  code,  debug,  test,  evalua- 
tion, and  integration  of  approved  changes,  and  documentation  amd  release 
I of  final  results.  They  establish  priorities  and  time  phase  all  tasks  and 

resources  required  for  the  program,  from  ECS  software  change  inception 


r through  final  completion  and  release  to  the  user.  These  personnel  also 

I serve  as  the  recognized  technical  expert  for  the  ECS  system  and  its 


interface  and  intef^ration  with  the  vreapon  system  and  all  supporting  elements. 
In  order  to  effectively  perform  these  assigned  responsibilities,  they  must 
possess  extensive  academic  and  professional  knowledge  of  scientific  and 
engineering  principles.  They  must  have  a thorough  and  detailed  knowledge 
of  the  3GS  software,  the  EGS  computer  and  the  weapon  system  to  include  all 
EGS  Interfaces  and  system  performance.  They  must  have  a detailed  knowledge 
of  real-time  software  operation;  laboratory  simulation;  host  computers  and 
system  mock-ups;  software  design;  configuration  management;  documentation; 
integration  testing;  evaluation;  verification  and  validation.  They  must 
also  possess  a working  knowled/'e  of  funding,  procurement,  and  weapon  system 
management;  and  must  be  capable  of  effectively  communicating  and  coordinat- 
ing through  the  vertical  and  horizontal  management  structure. 

Software  Engineers;  These  personnel  investigate  EGS  software  require- 
ments and  analyze  potential  solutions  considering  trade-off  analysis  involv- 
ing implementations,  cost,  algorithm  developments,  timing  requirements, 
memory  size,  hardware/software  integration,  and  support  equipment.  They 
translate  change  requirements  into  engineering  specifications;  design 
chaage  mechanisation  and  Integration;  develop  prorramming  code;  and  debug, 
test  and  document  results.  In  order  to  effectively  perform  these  assigned 
tasks,  they  must  have  a thorough  and  detailed  Itnowledge  of  digital  systems; 
EGS  software;  EGS  computers;  the  weapon  system  interfaces  and  performance; 
real-time  programming;  laboratory  support  systems;  math  modeling;  computer 
architecture  and  programming  languages. 

Staff  Analysts;  These  personnel  serve  as  mathematical  experts  for 
EGS  software  and  develop  numerical  and  mathematical  solutions  to  complex 
computer  programs  operating  in  real-time.  They  develop  mathematical  models 
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using  time  and  frequency  domain  transforms  and  apply  optimal  estimation 
theory  in  the  design  of  changes  to  Kalman  ajid  similar  statistical  filters. 
They  also  analyze  and  evaluate  complex  mathematical  models  used  in  such 
functions  as  navigation,  fire  control,  and  weapon  delivery;  and  collect, 
correlate  and  develop  statistical  performance  data.  In  order  to  effec- 
tively perform  these  tasks,  they  must  possess  the  s?~'“  detailed  system 
expertise  as  the  software  engineer,  as  well  as  being  uii  experienced  mathe- 
matician. 

Scientific  Programmers;  These  personnel  maintain  the  laboratory 
support  software  system.  They  design  and  develop  support  software  changes 
in  order  to  maintain  compatibility  with  the  ECS  software  changes,  and  to 
satisfy  requirements  for  special  test,  and  acquisition  software  programs. 
They  prepare  software  change  requirements  and  specifications,  taking  into 
consideration  host  computer  constraints,  computer  peripheral  interfaces 
and  compatibility  with  all  laboratory  equipment;  and  design,  code,  debug, 
test,  evaluate  and  document  final  changes  and  new  programs.  In  order  to 
effectively  perform  their  assigned  tasks,  they  must  possess  a detailed 
knowledge  of  the  support  software  system;  ECS  software;  host  computer 
systems;  simulators;  and  data  acquisition,  reduction,  and  analysis  systems. 
They  must  also  have  an  intimate  knowledge  of  real-time  programming;  mathe- 
matics; assemblers;  compilers;  peripheral  handlers;  acquisition  anc’  analy- 
sis software;  computer  architecture;  and  high  order,  assembly  and  machine 
language  programming.  They  should  also  be  familiaur  with  peripherals, 
digital  hardware  Interfaces,  and  hybrid  systems. 

Programmers ; These  personnel  prepare  implementation  specifications, 
design,  program,  trouble  shoot,  debug,  test  and  document  non-engineering 
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application  software  and  associated  laboratory  utility  and  off-line  miscel- 
laneous computer  programs.  They  provide  programming  support  to  the  scientific 
programmers,  analysts,  and  engineers;  and  must  be  capable  of  programming  in 
assembly  and  high  order  language. 

Test  and  Evaluation  Engineers;  These  personnel  serve  as  the  formal 
test  and  evaluation  team  for  ECS  software  modifications.  They  establish 
testing  requirements;  prepare  formal  test  plans;  develop  test  procedures; 
conduct  ECS  software  verification  and  validation;  and  prepare  the  test 
reports.  In  order  to  effectively  perform  their  assigned  tasks,  they  must 
possess  the  same  detailed  system  expertise  as  the  software  engineer. 

Configuration  Management;  These  personnel  develop,  establish,  imple- 
ment, and  enforce  regulations,  policies,  procedures  and  directives  required 
for  configuration  management  of  the  ECS  software  and  of  the  software  support 
system.  They  review,  control,  coordinate,  record  and  accoiuit  for  all  soft- 
ware changes.  They  assure  compliance  and  conforraity  with  configuration 
management  standards  and  formats  and  check  to  assure  that  all  software 
changes  are  controlled,  documented  and  compatible  with  the  weapon  system 
and  supporting  elements.  For  automated  configuration  management  systems, 
they  prepare  and  control  the  software  programs  which  generate  documentation. 
They  also  maintain  and  control  the  computer  data  base  software  libraries, 
and  storage  area.  In  order  to  perform  these  assigned  responsibilities, 
they  must  possess  detailed  knowledge  of  configuration  management  princi- 
ples, and  have  in-depth  knowl>5dge  of  software  and  computer  systems. 

Other  Personnel;  In  addition,  there  are  requirements  for  support 
from:  (l)  hardware  engineers  for  the  ECS  interf acting  subsystems,  and  for 
laboratory  hardware  equipment;  (z)  electronics  technicians  for  laboratory 
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equipment  maintenance;  (3)  technical  writers  to  prepare  final  enfineerinf 
and  technical  documentation  on  released  EG3  software;  and  (4;  draftsmen, 
clerical  and  supply  personnel. 

The  paramount  problem  impacting  £GS  softvrare  support  is  personnel . 
First,  there  are  no  appropriate  Givil  Service  classification  series  or 
military  career  fields  for  the  types  of  personnel  just  discussed.  This 
makes  it  extremely  difficult  to  acquire  and  retain  ,^ood  people.  Second, 
trained  and  experienced  personnel  are  in  short  supply,  both  in  DoD  and 
industry,  makinr  it  toup:h  to  find  good  people.  And  third,  because  of 
the  skill  levels  required,  training  is  long  and  expensive. 


2GUIH'IEI!T: 


Host  Gomputer  Systems;  These  are  the  general  purpose  computers  and 
peripherals,  i.e.,  magnetic  tape  units,  discs,  CRTs,  printers,  card 
readers,  paper  tape  punch,  etc.,  v;hich  are  used  to:  (l)  execute  the 

support  computer  programs;  (2)  operate  and  control  the  dynamic  simula- 


tors, computer  controlled  mock-ups,  and  data  acquisition  systems;  and 
(3)  perform  various  other  lo.boratory  functions  requirinf  the  use  of  a 


computer.  Jependin,'  on  the  EGS  software  support  requirements,  these 
computers  can  range  from  large  systems  like  an  IBM  36O/65  to  relatively 
small  and  inex-oenslve  mini  comnuters  like  a PDF  ll/40.  The  mini  computer 
approach  is  becoming  extremely  popular  for  several  reasons:  (l)  they 

are  relatively  Inexpensive  compared  to  the  large  computer  systems;  (,2) 
they  provide  ,‘-rreator  flexibility  and  versatility;  (3)  they  are  normally 
easier  to  modify  or  customize  for  specific  laborator  operations;  (4) 
they  are  easier  and  cheaper  to  maintain;  (5)  they  take  up  less  floor 
space;  (6)  if  a failure  occurs,  only  a part  of  the  operation  roes  down; 
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and  (?)  less  of  a queuing  problem  exists  when  operating  in  real-time,  i.e., 
driving  a dynamic  simulator.  Their  disadvantages  are:  (l)  many  mini  computer 

manufacturers  have  not  adequately  perfected  their  software;  (2)  customer 
support  is  sometimes  less  than  desirable;  and  (3)  for  large  ECS  computer 
systems,  they  may  not  have  the  capacity  to  meet  support  requirements. 

(15).  (17) 

Dynamic  Simulators:  These  are  hybrid  computer  systems  vihich  use  a 

host  computer  to  simulate  and  create  a dynamic  real  world  environment  in 
the  laboratory,  for  the  ECS  software;  and  is  accomplished  with  the  software 
resident  in  its  ECS  computer.  These  systems  are  configured  in  several  ways. 
One  configuration  uses  the  host  computer  to  simulate  all  ECS  computer  inputs. 
Another  configuration  uses  some  actual  weapon  system  equipment  and  stimulates 
this  equipment  with  host  computer  inputs;  and  other  configurations  use  some 
actual  vieapon  system  equipment  and  live  inputs.  Dynamic  simulators  have 
nroven  to  be  invaluable  development  and  test  tools  for  EG3  software.  Basi- 
cally, they  pTOvide  a capability  to:  (l)  generate  large  quantities  of  data, 

mission  scenarios  and  test  scenarios  as  inputs  to  the  EG3  software  under 
dynamic,  repeatable  and  controlled  conditions;  (2)  provide  man-in-the-loop 
or  automated  control  during  tost;  and  (3)  acquire  performance  data  on  the 
EG3  software  under  dynamic  test  conditions.  Figure  B-1  illustrates  the 
F-111  OFF  Dynamic  Simulator.  It  functions  as  follows:  The  aircraft  com- 

puters vfith  their  resident  OFF's,  shown  in  the  block  labeled  Mark  II  Digital 
Gomputer  Gomplex,  receive  inputs  from  the  aircraft  cockpit  mock-up  and  the 
host  computers,  and  output  control  signals  and  display  data  to  the  cock- 
pit. They  also  output  selected  acquisition  data  for  post  analysis.  The 
host  computer  generates  simulated  avionics  data  goin"  to  the  aircraft 
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computers;  generate  stimuli  for  the  avionics  housed  in  the  cockpit;  and 
generate  control  signals,  display  data,  and  operator  cues  going  to  the 
cockpit.  This  is  accomplished  by  using  computer  program  models  for  the 
aircraft  dynamics,  avionics,  external  world  and  navigation  reference 
systems.  The  cockpit  is  configured  very  similar  to  the  aircraft  cockpit 
and  has  actual  aircraft  controls  and  displays.  This  permits  realistic 
operator  inputs  and  control.  (l6),  (l4) 

System  Mock-ups;  Hock-ups  functionally  integrate,  in  the  laboratory, 
the  vieapon  system  equipment  which  interfaces  with  the  ECS  computers  in 
the  real  world.  For  an  aircraft  bomb/navigation  system,  this  equipment 
might  be  the  sensors,  cockpit  displays,  and  control  systems.  Mock-ups 
are  normally  configured  to  allow  easy  access  to  equipment  interfaces  and 
ease  of  instrumentation  and  monitoring  of  performance.  They  are  normally 
tied  to  the  host  computer  system  so  that  both  manual  and  automated  control 
can  be  achieved.  They  are  used  to  test  hardware/software  interfaces, 
timing,  compatibility,  RFI/EMI,  and  environmental  conditions.  They  are 
also  used  to  investigate  field  problems  when  it  has  not  been  deftermined 
whether  the  problem  is  hardvfare  or  software.  (l5)i  (14) 

Data  Acquisition  Systems;  These  are  systems  which  are  used  to  gather 
perfoimance  data  on  the  ECS  computer  program  during  stages  of  development 
and  testing.  They  differ  from  the  software  tools  used  for  verification, 
debw'  and  diagnostics  in  that  they  actually  monitor  the  ECS  software  as 
it  performs  in  the  ECS  computer.  These  systems  are  used  in  laboratory 
testing,  as  well  as  testing  in  actual  weapon  system  environments.  They 
generally  fall  into  two  categories:  (l)  systems  that  monitor  the  ECS 

computer  at  its  interface  (l/O);  and  (2)  systems  that  monitor  performance 
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internal  to  the  ECS  comnuter.  Instrunentation  systems  used  in  flifht 
test  are  examples  of  the  l/O  monitoring  system.  These  are  generally 
limited  to  validation  and  certification  testing.  The  latter  type  is 
designed  for  laboratory  use.  These  systems  provide  total  visibility  in- 
side the  ECS  computers  during  dynamic  real-time  program  execution.  They 
tie  to  the  internal  bus  structure  of  the  ECS  computer  and  capture  data  for 
full,  partial,  recent  history,  event  and  timing  traces.  These  types  of 
systems  can  be  used  in  all  phases  of  ECS  software  development,  but  are 
extremely  valuable  for  program  debug,  verification  and  validation.  (l6:<^‘l6) 

System  Resources;  This  is  the  weapon  system  equipment  required  in 
the  laboratory  to  support  ECS  softv/are  chanp'e  development  and  test.  This 
equipment  is  used  in  the  laboratory  mock-ups  and  dynamic  simulators.  In 
some  cases,  a complete  system  is  required  for  certification  testing,  e.g. , 
flight  test  aircraft  for  OFP's.  To  assure  test  results,  this  equipment 
should  be  of  production  quality  and  remain  in  an  operational  configuration. 
In  most  cases,  it  will  represent  the  largest  physical  resource  investment. 

SOmARE; 

ECS  Computer  Prorrams;  These  are  end  item  computer  programs  vrhich 
reside  in  weapon  system  operational  computers.  Firure  B-2  illustrates 
ECS  computer  programs  for  the  F3-111A  aircraft.  As  seen,  ECS  computer 
nro^rams  are  ret.era.lly  an  aggregate  of  subroutines  which  perform  multi- 
functions and  modes  for  the  weapon  system.  These  programs  a.re  subject 
to  continual  chaa-'^e  and  must  be  delivered  in  a state  and  format  that 
provides  for  case  of  maintenance  and  modification.  Changes  to  these 
programs  can  result  in  chan'*es  to  support  resources. 
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NAVIGATIOiJ-REU^TID  FUrJCTIOMS  AND  VVIAPON  DELIVERY  RELATED  FUNCTIONS 


CS  AREA  1 FAILURE 

• SRAM  DATA  CALCULATIONS  BACK-UP  IN 
EVENT  OF  GNC  OR  CS  AREA  1 FAILURE 


Frorraiumln,"  Aids;  These  are  assenblers,  compilers,  processors, 


editors,  etc. , which  execute  on  laboratory  host  computers  and  are  used 


by  profrrammors  and  engineers  as  development  tools.  Assemblers,  compilers. 


and  processors  permit  programming  changes  to  be  made  to  the  ECS  software 


in  assembly  and  high  order  language;  generate  documentation  such  as  memory 


maps,  absolute  listings  and  program  listings,  and  produce  EES  computer  pro- 


gram source  and  object  code  on  disc,  mag;netic  tape,  paper  tape,  etc. 


Editors  are  programmin,g  aids  which  peimit  on-line  interactive  text  edit- 


ing and  formatting  of  computer  program  changes  generally  from  CRT  temintals. 


Verification  Tools;  These  are  computer  programs  which  execute  on 


laboratory  host  computers  and  aid  in  debugging  program  code,  logic  and 


structure;  and  are  used  in  program  verification.  (l:5-^r)  These  programs 


include: 


Comparators:  Programs  which  do  an  instniction-by-instruction 


comnarison  on  tv;o  versions  of  the  same  program.  The  comparator  fla,gs 


differences,  thus  lndica.ting  where  a program  has  been  changed. 


Editors:  Programs  which  find  and  flag'  coding  errors  and  produce 


cross-reference  listings. 


Flowcharters:  Programs  vrhich  operate  on  a program  written  in 


either  assembly  or  higher  order  language  and  produce  a flowchart  of  the 


program.  This  flowchart  can  then  be  compared  to  the  original  program 


flowchart. 


Logic/Equation  Generators:  Programs  used  to  reconstruct  arith- 


metic text  and  to  flowchart  assembly  langua,"e  pro, '■'rams. 
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Pathfinders,  Traps  ajid  Traces s Programs  which  analyze  possible 
paths  through  a given  program.  This  information  is  useful  for  planning 
test  cases. 

Interpretive  Computer  Simulations  (ICS's) i Programs  which  perform 
an  instruction- level  simulation  of  the  ECS  computer.  The  host  computer 
is  programmed  to  act  on  the  ECS  computer  program  in  exactly  the  same  way 
as  the  ECS  computer  would  act.  The  host  computer  will  give  the  same  results 
bit-for-bit  as  would  be  produced  by  the  ECS  computer  under  the  same  con- 
ditions and  inputs.  The  advantage  of  am  ICS  simulation  is  greater  visi- 
bility and  diagnostics  during  program  debug. 

Host  Computer  Software;  This  software  consists  of:  (l)  the  host  com- 

puter operating  system,  i.e.,  the  program  which  remains  resident  in  the 
host  computer  and  controls  its  execution  and  operation;  (2)  host  computer 
diagnostics  which  are  used  to  trouble-shoot  the  host  computer  hardware 
and  operating  system;  (3)  peripheral  handlers  which  control  the  operation 
of  the  computer  peripherals,  i.e.,  tape  decks,  discs,  CRT's,  printers, 
readers,  etc.;  and  (4)  utility  programs  which  interface  and  link  the  host 
computer  and  operating  system  with  non-standard  handlers. 

Simulation  and  Test  Bed  Control  Software:  This  software  executes 

on  the  laboratory  host  computer  and  consists  of  real-time  and  non  real- 
time master  executive  programs  which  control  the  overall  operation  of 
dynamic  simulators  and  computer  controlled  test  bed  mock-ups.  This  soft- 
ware also  plans,  schedules,  prioritizes  and  controls  subservient  software 
such  as  equipment  models,  application  models,  displays,  programs  data  ac- 
quisition programs,  etc.  (16),  (lo) 


3 

i 

Ap-nlicatlon  3oftvfo.re;  This  aoftvjare  executes  on  the  latx^ratory  host  j 

conputer  and  is  nomally  a sub-set  of  the  dynanic  sinulatior.  and  conputcr  | 

controlled  mock-up  softuare.  These  conputer  nro-  rams  consist  of  weaj>on 
system  dynamics;  models  of  equipment  sub-systens  uhich  interface  with  or 
impact  on  the  2GS  computer  system;  extor:;al  vrorld  pi'orrams,  (e.g.  , atmos- 
phere, ea.rth,  terrain,  rre.vity,  reference  geometry,  targets,  reference 
systems,  etc,';  and  error,  noise  and  bias  models.  (l6),  (If) 

Display  Software : This  software  executes  on  the  laboratory  host  com- 

puter arid  is  used  to  renerate  specia.1  c;raphical  displays  in  real  and  non- 
real  tine,  (e.-.,  displays  of  ta-rpctirr  '-oometry  and  simulated  position  i. 

and  attitude  of  the  vreapor.  system),  ionially,  this  softwa.re  displa,ys  data 
both  in  empirica.l  and  .-ranhical  formats.  (l6),  (l^) 

Data  Acquisition  3oftwa.re;  This  software  consists  of  data  acquisi- 
tion, red’i.ction  and  a,na,lyEis  programs.  Acquisitiori  pro' rams  control  the 
operation  o:^  the  instr'ur.cntatlor.  and.  data  acquisition  system,  foma.t  and  * 

temporarily  store  data.  Depending  on  thic  acquisition  device,  this  soft- 
•wa,rc  : a,y  reside  in  the  acquisition  system,  host  co.mputer,  the  EGD  computer, 
or  some  combi.na.tior . Data  reduction  programs  execute  on  thie  host  computer 
a.nd  :"'ormat,  convert,  tLmc  ta  , cor3’ela,te  and  produce  the  requires^  computer 
outputs.  Analysis  program.s  a.lsc  execute  on  the  host  computer  amd  evaluate 
reduced  data  a-ainst  defined  criteria  a.nd  display  or  print  results.  (22) 

Gon.fifcu.ra.tlon  hanagement  Doftwarc;  This  software  executes  on  the 
laV-ora.tory  host  computer  and  con.si.sts  o":  (l)  docuTier.ta.tion  generators 

which  ta.bulatc  DGG  computer  progra.m  change  requirements  a.rd  ger.era.te  speci- 
fication data,  source  a.n.d  object  code,  narra.tivoc , ba.sclirrc  co:'.riguratioi.c 
and  other  con^’iguratior.  da.t-’  o;.  each  TG.G  corgputcr  program  change; 


(2)  documentation  processors  vfhich  define,  standardize  and  format  soft- 
ware documentation;  and  (3)  data  base  control  and  status  accounting  computer 
programs  which  maintain  listings  and  records  of  all  software  and  softvjare 
documentation  to  include  status  and  configuration  identification,  (l?) 

Figure  B-3  illustrates  the  support  software  system  used  in  develop- 
ment of  F-14  aircraft  weapon  control  computer  prograjns.  The  figure  is 
helpful  in  illustrating  the  complexity  and  inter-relationship  of  support 
software  for  BCS  computer  programs. 
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AwO-»  SYSTfM  DESIGN  «IQUI«EMENTS 


AWG-9  program  dcvclopnicnt  process 

The  support  software  system  is  utilized  in  all  five 
design  pliases  of  program  development 

SUITORT  C0I'T.:A1;K 
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